Drug development is a complex and long process that involves screening thousands of drug candidates and using computational or experimental methods to evaluate leads. According to McKinsey, a single drug can take 10 years and cost an average of $2.6 billion to go through disease target identification, drug screening, drug-target validation, and eventual commercial launch. […]
( 15
min )
If you are a business analyst, understanding customer behavior is probably one of the most important things you care about. Understanding the reasons and mechanisms behind customer purchase decisions can facilitate revenue growth. However, the loss of customers (commonly referred to as customer churn) always poses a risk. Gaining insights into why customers leave can […]
( 14
min )
In an age where efficiency is king, manufacturing firms are in a constant race to outshine their competition. Imagine if you could boost productivity, slash downtime, and cut costs all at once. Sounds like a dream, right? The good news is, this isn’t a fantasy. It’s achievable through Internet of Things (IoT) solutions. IoT solutions… Read More »Increase efficiency of manufacturing operations with IoT solutions
The post Increase efficiency of manufacturing operations with IoT solutions appeared first on Data Science Central.
( 21
min )
“If you start by creating your data, then it’s like you are piling up some value or you’re creating some assets,” WordLift CEO Andrea Volpini told me in our recent FAIR Data Forecast interview. Volpini’s an advocate for adding structured data such as Schema.org to your content. That way, the content becomes logically connected and… Read More »Human-centered data networking with interpersonal knowledge graphs
The post Human-centered data networking with interpersonal knowledge graphs appeared first on Data Science Central.
( 21
min )
“PhotoGuard,” developed by MIT CSAIL researchers, prevents unauthorized image manipulation, safeguarding authenticity in the era of advanced generative models.
( 10
min )
One of the reasons that I moved back to Iowa last year was that I saw an opportunity to work with local educational institutions to create an AI Institute for organizations in middle America that either get overlooked in the AI conversation or are unsure what AI means to them. I wanted to reduce the… Read More »Introduction to “AI & Data Literacy: Empowering Citizens of Data Science”
The post Introduction to “AI & Data Literacy: Empowering Citizens of Data Science” appeared first on Data Science Central.
( 22
min )
Decoding EEG signals for imagined speech is a challenging task due to the
high-dimensional nature of the data and low signal-to-noise ratio. In recent
years, denoising diffusion probabilistic models (DDPMs) have emerged as
promising approaches for representation learning in various domains. Our study
proposes a novel method for decoding EEG signals for imagined speech using
DDPMs and a conditional autoencoder named Diff-E. Results indicate that Diff-E
significantly improves the accuracy of decoding EEG signals for imagined speech
compared to traditional machine learning techniques and baseline models. Our
findings suggest that DDPMs can be an effective tool for EEG signal decoding,
with potential implications for the development of brain-computer interfaces
that enable communication through imagined speech.
( 2
min )
Optimization methods are essential in solving complex problems across various
domains. In this research paper, we introduce a novel optimization method
called Gaussian Crunching Search (GCS). Inspired by the behaviour of particles
in a Gaussian distribution, GCS aims to efficiently explore the solution space
and converge towards the global optimum. We present a comprehensive analysis of
GCS, including its working mechanism, and potential applications. Through
experimental evaluations and comparisons with existing optimization methods, we
highlight the advantages and strengths of GCS. This research paper serves as a
valuable resource for researchers, practitioners, and students interested in
optimization, providing insights into the development and potential of Gaussian
Crunching Search as a new and promising approach.
( 2
min )
We prove that black-box variational inference (BBVI) with control variates,
particularly the sticking-the-landing (STL) estimator, converges at a geometric
(traditionally called "linear") rate under perfect variational family
specification. In particular, we prove a quadratic bound on the gradient
variance of the STL estimator, one which encompasses misspecified variational
families. Combined with previous works on the quadratic variance condition,
this directly implies convergence of BBVI with the use of projected stochastic
gradient descent. We also improve existing analysis on the regular closed-form
entropy gradient estimators, which enables comparison against the STL estimator
and provides explicit non-asymptotic complexity guarantees for both.
( 2
min )
This paper studies the estimation and inference of treatment histories in
panel data settings when treatments change dynamically over time.
We propose a method that allows for (i) treatments to be assigned dynamically
over time based on high-dimensional covariates, past outcomes and treatments;
(ii) outcomes and time-varying covariates to depend on treatment trajectories;
(iii) heterogeneity of treatment effects.
Our approach recursively projects potential outcomes' expectations on past
histories. It then controls the bias by balancing dynamically observable
characteristics. We study the asymptotic and numerical properties of the
estimator and illustrate the benefits of the procedure in an empirical
application.
( 2
min )
In various fields, such as traffic management, law enforcement, and parking management, license plate recognition is a crucial application of computer vision that is used to analyze license plates. In this article, we will review the Chinese City Parking Dataset (CCPD), which is one of the most widely used computer vision datasets for tasks that… Read More »Understanding license plate recognition with the CCPD computer vision datasets
The post Understanding license plate recognition with the CCPD computer vision datasets appeared first on Data Science Central.
( 20
min )
From smart factories to next-generation railway systems, developers and enterprises across the world are racing to fuel industrial digitalization opportunities at every scale. Key to this is the open-source Universal Scene Description (USD) framework, or OpenUSD, along with metaverse applications powered by AI. OpenUSD, originally developed by Pixar for large-scale feature film pipelines for animation Read article >
( 7
min )
AI is improving ways to power the world by tapping the sun and the wind, along with cutting-edge technologies. The latest episode in the I AM AI video series showcases how artificial intelligence can help optimize solar and wind farms, simulate climate and weather, enhance power grid reliability and resilience, advance carbon capture and power Read article >
( 6
min )
Get ready for Gunfire Games and Gearbox Publishing’s highly anticipated Remnant II, available for members to stream on GeForce NOW at launch. It leads eight new games coming to the cloud gaming platform. Ultimate and Priority members, make sure to grab the Guild Wars 2 rewards, available now through Thursday, Aug. 31. Visit the GeForce Read article >
( 5
min )
Managing server failures at the scale of a cloud platform is challenging. The Hyrax fail-in-place approach reduces the need for immediate repairs and creates a path toward lowering water consumption and carbon emissions in cloud datacenters.
The post A fail-in-place approach for sustainable server operations appeared first on Microsoft Research.
( 12
min )
Today we are excited to announce that Stable Diffusion XL 1.0 (SDXL 1.0) is available for customers through Amazon SageMaker JumpStart. SDXL 1.0 is the latest image generation model from Stability AI. SDXL 1.0 enhancements include native 1024-pixel image generation at a variety of aspect ratios. It’s designed for professional use, and calibrated for high-resolution […]
( 12
min )
The increase in online social activities such as social networking or online gaming is often riddled with hostile or aggressive behavior that can lead to unsolicited manifestations of hate speech, cyberbullying, or harassment. For example, many online gaming communities offer voice chat functionality to facilitate communication among their users. Although voice chat often supports friendly […]
( 8
min )
Generative AI models have been experiencing rapid growth in recent months due to its impressive capabilities in creating realistic text, images, code, and audio. Among these models, Stable Diffusion models stand out for their unique strength in creating high-quality images based on text prompts. Stable Diffusion can generate a wide variety of high-quality images, including […]
( 12
min )
Breakthroughs in artificial intelligence (AI) and machine learning (ML) have been in the headlines for months—and for good reason. The emerging and evolving capabilities of this technology promises new business opportunities for customer across all sectors and industries. But the speed of this revolution has made it harder for organizations and consumers to assess what […]
( 6
min )
Generative AI Foundations on AWS is a new technical deep dive course that gives you the conceptual fundamentals, practical advice, and hands-on guidance to pre-train, fine-tune, and deploy state-of-the-art foundation models on AWS and beyond. Developed by AWS generative AI worldwide foundations lead Emily Webber, this free hands-on course and the supporting GitHub source code […]
( 6
min )
As a pioneer in artificial intelligence and machine learning, AWS is committed to developing and deploying generative AI responsibly As one of the most transformational innovations of our time, generative AI continues to capture the world’s imagination, and we remain as committed as ever to harnessing it responsibly. With a team of dedicated responsible AI […]
( 5
min )
AWS users can now access the leading performance demonstrated in industry benchmarks of AI training and inference. The cloud giant officially switched on a new Amazon EC2 P5 instance powered by NVIDIA H100 Tensor Core GPUs. The service lets users scale generative AI, high performance computing (HPC) and other applications with a click from a Read article >
( 6
min )
The world increasingly runs on code. Accelerating the work of those who create that code will boost their productivity — and that’s just what AI startup Codeium, a member of NVIDIA’s Inception program for startups, aims to do. On the latest episode of NVIDIA’s AI Podcast, host Noah Kravitz interviewed Codeium founder and CEO Varun Read article >
( 5
min )
Researchers develop a machine-learning technique that can efficiently learn to control a robot, leading to better performance with fewer data.
( 10
min )
Welcome to the exciting world of digital marketing! In this blog, we’ll delve into this thrilling frontier where optimization meets automation and Artificial Intelligence is at the center. No longer must manual labor and guesswork play an essential part in developing effective marketing strategies; with AI’s capabilities now at their disposal, marketers with digital presence… Read More »From automation to optimization: How AI is revolutionizing digital marketing campaigns
The post From automation to optimization: How AI is revolutionizing digital marketing campaigns appeared first on Data Science Central.
( 24
min )
With recent advancements in generative AI, there are lot of discussions happening on how to use generative AI across different industries to solve specific business problems. Generative AI is a type of AI that can create new content and ideas, including conversations, stories, images, videos, and music. It is all backed by very large models […]
( 9
min )
NVIDIA DGX Cloud — which delivers tools that can turn nearly any company into an AI company — is now broadly available, with thousands of NVIDIA GPUs online on Oracle Cloud Infrastructure, as well as NVIDIA infrastructure located in the U.S. and U.K. Unveiled at NVIDIA’s GTC conference in March, DGX Cloud is an AI Read article >
( 5
min )
We’re gonna need a bigger boat this week In the NVIDIA Studio as Alessandro Mastronardi, senior artist and programmer at BBC Studios, shares heart-stopping shark videos and renders.
( 7
min )
This blog post was co-authored, and includes an introduction, by Zilong Bai, senior natural language processing engineer at Patsnap. You’re likely familiar with the autocomplete suggestion feature when you search for something on Google or Amazon. Although the search terms in these scenarios are pretty common keywords or expressions that we use in daily life, […]
( 9
min )
When deploying Deep Learning models at scale, it is crucial to effectively utilize the underlying hardware to maximize performance and cost benefits. For production workloads requiring high throughput and low latency, the selection of the Amazon Elastic Compute Cloud (EC2) instance, model serving stack, and deployment architecture is very important. Inefficient architecture can lead to […]
( 15
min )
The dataset, being collected as part of a US Coast Guard science mission, will be released open source to help advance naval mission planning and climate change studies.
( 9
min )
In this blog, I will now focus on generative AI megatrends. By that, I mean, trends and underlying trends that could be big in the future – focusing on the technology of LLM but also the wider impact of LLMs on the economy and society. I will hence identify and follow some key trends –… Read More »Generative AI megatrends: Are companies using the excuse of AI to get rid of jobs?
The post Generative AI megatrends: Are companies using the excuse of AI to get rid of jobs? appeared first on Data Science Central.
( 19
min )
Partial differential equations (PDEs) are important tools to model physical
systems and including them into machine learning models is an important way of
incorporating physical knowledge. Given any system of linear PDEs with constant
coefficients, we propose a family of Gaussian process (GP) priors, which we
call EPGP, such that all realizations are exact solutions of this system. We
apply the Ehrenpreis-Palamodov fundamental principle, which works as a
non-linear Fourier transform, to construct GP kernels mirroring standard
spectral methods for GPs. Our approach can infer probable solutions of linear
PDE systems from any data such as noisy measurements, or pointwise defined
initial and boundary conditions. Constructing EPGP-priors is algorithmic,
generally applicable, and comes with a sparse version (S-EPGP) that learns the
relevant spectral frequencies and works better for big data sets. We
demonstrate our approach on three families of systems of PDEs, the heat
equation, wave equation, and Maxwell's equations, where we improve upon the
state of the art in computation time and precision, in some experiments by
several orders of magnitude.
( 3
min )
Labeling of multivariate biomedical time series data is a laborious and
expensive process. Self-supervised contrastive learning alleviates the need for
large, labeled datasets through pretraining on unlabeled data. However, for
multivariate time series data, the set of input channels often varies between
applications, and most existing work does not allow for transfer between
datasets with different sets of input channels. We propose learning one encoder
to operate on all input channels individually. We then use a message passing
neural network to extract a single representation across channels. We
demonstrate the potential of this method by pretraining our model on a dataset
with six EEG channels and then fine-tuning it on a dataset with two different
EEG channels. We compare models with and without the message passing neural
network across different contrastive loss functions. We show that our method,
combined with the TS2Vec loss, outperforms all other methods in most settings.
( 2
min )
Centrality metrics are vital for network analysis, but selecting the most
appropriate measures for specific applications remains challenging among the
400+ proposed indices. Existing approaches -- model-based, data-driven, and
axiomatic -- have limitations. To address this, we introduce the culling
method, leveraging expert preferences regarding centrality behavior on simple
graphs. It involves forming a set of candidate measures, generating a list of
as small graphs as possible needed to ``separate'' measures from each other,
constructing a decision-tree survey, and identifying the measure consistent
with expert responses. We apply this method to a diverse set of 40
centralities, including new kernel-based measures, and combine it with the
axiomatic approach. Remarkably, only 13 small 1-trees suffice to separate all
40 measures, among which there are pairs of close ones. The culling method
offers a low-cost solution in terms of labor and time, complements existing
methods for measure selection, and reveals important peculiarities of
centrality measures.
( 2
min )
We consider the problem of learning from data corrupted by
underrepresentation bias, where positive examples are filtered from the data at
different, unknown rates for a fixed number of sensitive groups. We show that
with a small amount of unbiased data, we can efficiently estimate the
group-wise drop-out parameters, even in settings where intersectional group
membership makes learning each intersectional rate computationally infeasible.
Using this estimate for the group-wise drop-out rate, we construct a
re-weighting scheme that allows us to approximate the loss of any hypothesis on
the true distribution, even if we only observe the empirical error on a biased
sample. Finally, we present an algorithm encapsulating this learning and
re-weighting process, and we provide strong PAC-style guarantees that, with
high probability, our estimate of the risk of the hypothesis over the true
distribution will be arbitrarily close to the true risk.
( 2
min )
Adversarial examples are inputs to machine learning models that an attacker
has intentionally designed to confuse the model into making a mistake. Such
examples pose a serious threat to the applicability of machine-learning-based
systems, especially in life- and safety-critical domains. To address this
problem, the area of adversarial robustness investigates mechanisms behind
adversarial attacks and defenses against these attacks. This survey reviews a
particular subset of this literature that focuses on investigating properties
of training data in the context of model robustness under evasion attacks. It
first summarizes the main properties of data leading to adversarial
vulnerability. It then discusses guidelines and techniques for improving
adversarial robustness by enhancing the data representation and learning
procedures, as well as techniques for estimating robustness guarantees given
particular data. Finally, it discusses gaps of knowledge and promising future
research directions in this area.
( 2
min )
Robust reinforcement learning (RL) aims to find a policy that optimizes the
worst-case performance in the face of uncertainties. In this paper, we focus on
action robust RL with the probabilistic policy execution uncertainty, in which,
instead of always carrying out the action specified by the policy, the agent
will take the action specified by the policy with probability $1-\rho$ and an
alternative adversarial action with probability $\rho$. We establish the
existence of an optimal policy on the action robust MDPs with probabilistic
policy execution uncertainty and provide the action robust Bellman optimality
equation for its solution. Furthermore, we develop Action Robust Reinforcement
Learning with Certificates (ARRLC) algorithm that achieves minimax optimal
regret and sample complexity. Furthermore, we conduct numerical experiments to
validate our approach's robustness, demonstrating that ARRLC outperforms
non-robust RL algorithms and converges faster than the robust TD algorithm in
the presence of action perturbations.
( 2
min )
Attention is the core mechanism of today's most used architectures for
natural language processing and has been analyzed from many perspectives,
including its effectiveness for machine translation-related tasks. Among these
studies, attention resulted to be a useful source of information to get
insights about word alignment also when the input text is substituted with
audio segments, as in the case of the speech translation (ST) task. In this
paper, we propose AlignAtt, a novel policy for simultaneous ST (SimulST) that
exploits the attention information to generate source-target alignments that
guide the model during inference. Through experiments on the 8 language pairs
of MuST-C v1.0, we show that AlignAtt outperforms previous state-of-the-art
SimulST policies applied to offline-trained models with gains in terms of BLEU
of 2 points and latency reductions ranging from 0.5s to 0.8s across the 8
languages.
( 2
min )
We propose simple nonparametric estimators for mediated and time-varying dose
response curves based on kernel ridge regression. By embedding Pearl's
mediation formula and Robins' g-formula with kernels, we allow treatments,
mediators, and covariates to be continuous in general spaces, and also allow
for nonlinear treatment-confounder feedback. Our key innovation is a
reproducing kernel Hilbert space technique called sequential kernel embedding,
which we use to construct simple estimators for complex causal estimands. Our
estimators preserve the generality of classic identification while also
achieving nonasymptotic uniform rates. In nonlinear simulations with many
covariates, we demonstrate strong performance. We estimate mediated and
time-varying dose response curves of the US Job Corps, and clean data that may
serve as a benchmark in future work. We extend our results to mediated and
time-varying treatment effects and counterfactual distributions, verifying
semiparametric efficiency and weak convergence.
( 2
min )
The drastic growth of electric vehicles and photovoltaics can introduce new
challenges, such as electrical current congestion and voltage limit violations
due to peak load demands. These issues can be mitigated by controlling the
operation of electric vehicles i.e., smart charging. Centralized smart charging
solutions have already been proposed in the literature. But such solutions may
lack scalability and suffer from inherent drawbacks of centralization, such as
a single point of failure, and data privacy concerns. Decentralization can help
tackle these challenges. In this paper, a fully decentralized smart charging
system is proposed using the philosophy of adaptive multi-agent systems. The
proposed system utilizes multi-armed bandit learning to handle uncertainties in
the system. The presented system is decentralized, scalable, real-time,
model-free, and takes fairness among different players into account. A detailed
case study is also presented for performance evaluation.
( 2
min )
Time series anomaly detection is crucial for industrial monitoring services
that handle a large volume of data, aiming to ensure reliability and optimize
system performance. Existing methods often require extensive labeled resources
and manual parameter selection, highlighting the need for automation. This
paper proposes a comprehensive framework for automatic parameter optimization
in time series anomaly detection models. The framework introduces three
optimization targets: prediction score, shape score, and sensitivity score,
which can be easily adapted to different model backbones without prior
knowledge or manual labeling efforts. The proposed framework has been
successfully applied online for over six months, serving more than 50,000 time
series every minute. It simplifies the user's experience by requiring only an
expected sensitive value, offering a user-friendly interface, and achieving
desired detection results. Extensive evaluations conducted on public datasets
and comparison with other methods further confirm the effectiveness of the
proposed framework.
( 2
min )
Performance of a pre-trained semantic segmentation model is likely to
substantially decrease on data from a new domain. We show a pre-trained model
can be adapted to unlabelled target domain data by calculating soft-label
prototypes under the domain shift and making predictions according to the
prototype closest to the vector with predicted class probabilities. The
proposed adaptation procedure is fast, comes almost for free in terms of
computational resources and leads to considerable performance improvements. We
demonstrate the benefits of such label calibration on the highly-practical
synthetic-to-real semantic segmentation problem.
( 2
min )
Granger causality (GC) is often considered not an actual form of causality.
Still, it is arguably the most widely used method to assess the predictability
of a time series from another one. Granger causality has been widely used in
many applied disciplines, from neuroscience and econometrics to Earth sciences.
We revisit GC under a graphical perspective of state-space models. For that, we
use GraphEM, a recently presented expectation-maximisation algorithm for
estimating the linear matrix operator in the state equation of a
linear-Gaussian state-space model. Lasso regularisation is included in the
M-step, which is solved using a proximal splitting Douglas-Rachford algorithm.
Experiments in toy examples and challenging climate problems illustrate the
benefits of the proposed model and inference technique over standard Granger
causality methods.
( 2
min )
We present FACADE, a novel probabilistic and geometric framework designed for
unsupervised mechanistic anomaly detection in deep neural networks. Its primary
goal is advancing the understanding and mitigation of adversarial attacks.
FACADE aims to generate probabilistic distributions over circuits, which
provide critical insights to their contribution to changes in the manifold
properties of pseudo-classes, or high-dimensional modes in activation space,
yielding a powerful tool for uncovering and combating adversarial attacks. Our
approach seeks to improve model robustness, enhance scalable model oversight,
and demonstrates promising applications in real-world deployment settings.
( 2
min )
In this work we introduce $\nu^2$-Flows, an extension of the $\nu$-Flows
method to final states containing multiple neutrinos. The architecture can
natively scale for all combinations of object types and multiplicities in the
final state for any desired neutrino multiplicities. In $t\bar{t}$ dilepton
events, the momenta of both neutrinos and correlations between them are
reconstructed more accurately than when using the most popular standard
analytical techniques, and solutions are found for all events. Inference time
is significantly faster than competing methods, and can be reduced further by
evaluating in parallel on graphics processing units. We apply $\nu^2$-Flows to
$t\bar{t}$ dilepton events and show that the per-bin uncertainties in unfolded
distributions is much closer to the limit of performance set by perfect
neutrino reconstruction than standard techniques. For the chosen double
differential observables $\nu^2$-Flows results in improved statistical
precision for each bin by a factor of 1.5 to 2 in comparison to the Neutrino
Weighting method and up to a factor of four in comparison to the Ellipse
approach.
( 2
min )
Data-driven optimization uses contextual information and machine learning
algorithms to find solutions to decision problems with uncertain parameters.
While a vast body of work is dedicated to interpreting machine learning models
in the classification setting, explaining decision pipelines involving learning
algorithms remains unaddressed. This lack of interpretability can block the
adoption of data-driven solutions as practitioners may not understand or trust
the recommended decisions. We bridge this gap by introducing a counterfactual
explanation methodology tailored to explain solutions to data-driven problems.
We introduce two classes of explanations and develop methods to find nearest
explanations of random forest and nearest-neighbor predictors. We demonstrate
our approach by explaining key problems in operations management such as
inventory management and routing.
( 2
min )
In this article we describe an efficient approach to guiding language model
text generation with regular expressions and context-free grammars. Our
approach adds little to no overhead to the token sequence generation process,
and makes guided generation feasible in practice. An implementation is provided
in the open source Python library Outlines.
( 2
min )
The research on code-mixed data is limited due to the unavailability of
dedicated code-mixed datasets and pre-trained language models. In this work, we
focus on the low-resource Indian language Marathi which lacks any prior work in
code-mixing. We present L3Cube-MeCorpus, a large code-mixed Marathi-English
(Mr-En) corpus with 10 million social media sentences for pretraining. We also
release L3Cube-MeBERT and MeRoBERTa, code-mixed BERT-based transformer models
pre-trained on MeCorpus. Furthermore, for benchmarking, we present three
supervised datasets MeHate, MeSent, and MeLID for downstream tasks like
code-mixed Mr-En hate speech detection, sentiment analysis, and language
identification respectively. These evaluation datasets individually consist of
manually annotated \url{~}12,000 Marathi-English code-mixed tweets. Ablations
show that the models trained on this novel corpus significantly outperform the
existing state-of-the-art BERT models. This is the first work that presents
artifacts for code-mixed Marathi research. All datasets and models are publicly
released at https://github.com/l3cube-pune/MarathiNLP .
( 2
min )
Active learning is a well-studied approach to learning formal specifications,
such as automata. In this work, we extend active specification learning by
proposing a novel framework that strategically requests a combination of
membership labels and pair-wise preferences, a popular alternative to
membership labels. The combination of pair-wise preferences and membership
labels allows for a more flexible approach to active specification learning,
which previously relied on membership labels only. We instantiate our framework
in two different domains, demonstrating the generality of our approach. Our
results suggest that learning from both modalities allows us to robustly and
conveniently identify specifications via membership and preferences.
( 2
min )
This note describes a new approach to classifying graphs that leverages graph
generative models (GGM). Assuming a GGM that defines a joint probability
distribution over graphs and their class labels, I derive classification
formulas for the probability of a class label given a graph. A new conditional
ELBO can be used to train a generative graph auto-encoder model for
discrimination. While leveraging generative models for classification has been
well explored for non-relational i.i.d. data, to our knowledge it is a novel
approach to graph classification.
( 2
min )
Flow map learning (FML), in conjunction with deep neural networks (DNNs), has
shown promises for data driven modeling of unknown dynamical systems. A
remarkable feature of FML is that it is capable of producing accurate
predictive models for partially observed systems, even when their exact
mathematical models do not exist. In this paper, we present an overview of the
FML framework, along with the important computational details for its
successful implementation. We also present a set of well defined benchmark
problems for learning unknown dynamical systems. All the numerical details of
these problems are presented, along with their FML results, to ensure that the
problems are accessible for cross-examination and the results are reproducible.
( 2
min )
Deep Learners (DLs) are the state-of-art predictive mechanism with
applications in many fields requiring complex high dimensional data processing.
Although conventional DLs get trained via gradient descent with
back-propagation, Kalman Filter (KF)-based techniques that do not need gradient
computation have been developed to approximate DLs. We propose a multi-arm
extension of a KF-based DL approximator that can mimic DL when the sample size
is too small to train a multi-arm DL. The proposed Matrix Ensemble Kalman
Filter-based multi-arm ANN (MEnKF-ANN) also performs explicit model stacking
that becomes relevant when the training sample has an unequal-size feature set.
Our proposed technique can approximate Long Short-term Memory (LSTM) Networks
and attach uncertainty to the predictions obtained from these LSTMs with
desirable coverage. We demonstrate how MEnKF-ANN can "adequately" approximate
an LSTM network trained to classify what carbohydrate substrates are digested
and utilized by a microbiome sample whose genomic sequences consist of
polysaccharide utilization loci (PULs) and their encoded genes.
( 2
min )
Source-free domain adaptation has become popular because of its practical
usefulness and no need to access source data. However, the adaptation process
still takes a considerable amount of time and is predominantly based on
optimization that relies on back-propagation. In this work we present a simple
feed-forward approach that challenges the need for back-propagation based
adaptation. Our approach is based on computing prototypes of classes under the
domain shift using a pre-trained model. It achieves strong improvements in
accuracy compared to the pre-trained model and requires only a small fraction
of time of existing domain adaptation methods.
( 2
min )
We propose simple nonparametric estimators for mediated and time-varying dose
response curves based on kernel ridge regression. By embedding Pearl's
mediation formula and Robins' g-formula with kernels, we allow treatments,
mediators, and covariates to be continuous in general spaces, and also allow
for nonlinear treatment-confounder feedback. Our key innovation is a
reproducing kernel Hilbert space technique called sequential kernel embedding,
which we use to construct simple estimators for complex causal estimands. Our
estimators preserve the generality of classic identification while also
achieving nonasymptotic uniform rates. In nonlinear simulations with many
covariates, we demonstrate strong performance. We estimate mediated and
time-varying dose response curves of the US Job Corps, and clean data that may
serve as a benchmark in future work. We extend our results to mediated and
time-varying treatment effects and counterfactual distributions, verifying
semiparametric efficiency and weak convergence.
( 2
min )
The tuning of stochastic gradient algorithms (SGAs) for optimization and
sampling is often based on heuristics and trial-and-error rather than
generalizable theory. We address this theory--practice gap by characterizing
the large-sample statistical asymptotics of SGAs via a joint
step-size--sample-size scaling limit. We show that iterate averaging with a
large fixed step size is robust to the choice of tuning parameters and
asymptotically has covariance proportional to that of the MLE sampling
distribution. We also prove a Bernstein--von Mises-like theorem to guide
tuning, including for generalized posteriors that are robust to model
misspecification. Numerical experiments validate our results and
recommendations in realistic finite-sample regimes. Our work lays the
foundation for a systematic analysis of other stochastic gradient Markov chain
Monte Carlo algorithms for a wide range of models.
( 2
min )
Rodents such as rats and mice are associated with a number of health risks and are known to spread more than 35 diseases. Identifying regions of high rodent activity can help local authorities and pest control organizations plan for interventions effectively and exterminate the rodents. In this post, we show how to monitor and visualize […]
( 7
min )
Microsoft Research is proud to be a sponsor of ICML 2023! From audio classification to privacy estimation and more, explore conference highlights in our latest blog post.
The post Microsoft at ICML 2023: Discoveries and advancements in machine learning appeared first on Microsoft Research.
( 10
min )
This is a guest post by Mario Namtao Shianti Larcher, Head of Computer Vision at Enel. Enel, which started as Italy’s national entity for electricity, is today a multinational company present in 32 countries and the first private network operator in the world with 74 million users. It is also recognized as the first renewables […]
( 8
min )
Artificial intelligence (AI) has become an important and popular topic in the technology community. As AI has evolved, we have seen different types of machine learning (ML) models emerge. One approach, known as ensemble modeling, has been rapidly gaining traction among data scientists and practitioners. In this post, we discuss what ensemble models are and […]
( 12
min )
It’s a party this GFN Thursday with several newly launched titles streaming on GeForce NOW. Revel in gaming goodness with Xenonauts 2, Viewfinder and Techtonica, among the four new games joining the cloud this week. Portal fans, stay tuned — the Portal: Prelude RTX mod will be streaming on GeForce NOW to members soon. Plus, Read article >
( 5
min )
For over a decade, Xbox has been leveraging AI to elevate gaming. Haiyan Zhang, GM of Gaming AI, explores the collaborations behind the work and the potential for generative AI to support better experiences for both players and game creators.
The post Collaborators: Gaming AI with Haiyan Zhang appeared first on Microsoft Research.
( 29
min )
Large language models (LLMs) can be used to analyze complex documents and provide summaries and answers to questions. The post Domain-adaptation Fine-tuning of Foundation Models in Amazon SageMaker JumpStart on Financial data describes how to fine-tune an LLM using your own dataset. Once you have a solid LLM, you’ll want to expose that LLM to […]
( 7
min )
Amazon SageMaker Model Cards enable you to standardize how models are documented, thereby achieving visibility into the lifecycle of a model, from designing, building, training, and evaluation. Model cards are intended to be a single source of truth for business and technical metadata about the model that can reliably be used for auditing and documentation […]
( 7
min )
Saildrone is making a splash in autonomous oceanic monitoring. The startup’s nautical data collection technology has tracked hurricanes up close in the North Atlantic, discovered a 3,200-foot underwater mountain in the Pacific Ocean and begun to help map the entirety of the world’s ocean floor. Based in the San Francisco Bay Area, the company develops Read article >
( 6
min )
Undetected partial discharges (PDs) are a safety critical issue in high
voltage (HV) gas insulated systems (GIS). While the diagnosis of PDs under AC
voltage is well-established, the analysis of PDs under DC voltage remains an
active research field. A key focus of these investigations is the
classification of different PD sources to enable subsequent sophisticated
analysis.
In this paper, we propose and analyze a neural network-based approach for
classifying PD signals caused by metallic protrusions and conductive particles
on the insulator of HVDC GIS, without relying on pulse sequence analysis
features. In contrast to previous approaches, our proposed model can
discriminate the studied PD signals obtained at negative and positive
potentials, while also generalizing to unseen operating voltage multiples.
Additionally, we compare the performance of time- and frequency-domain input
signals and explore the impact of different normalization schemes to mitigate
the influence of free-space path loss between the sensor and defect location.
( 2
min )
Determining clinically relevant physiological states from multivariate time
series data with missing values is essential for providing appropriate
treatment for acute conditions such as Traumatic Brain Injury (TBI),
respiratory failure, and heart failure. Utilizing non-temporal clustering or
data imputation and aggregation techniques may lead to loss of valuable
information and biased analyses. In our study, we apply the SLAC-Time
algorithm, an innovative self-supervision-based approach that maintains data
integrity by avoiding imputation or aggregation, offering a more useful
representation of acute patient states. By using SLAC-Time to cluster data in a
large research dataset, we identified three distinct TBI physiological states
and their specific feature profiles. We employed various clustering evaluation
metrics and incorporated input from a clinical domain expert to validate and
interpret the identified physiological states. Further, we discovered how
specific clinical events and interventions can influence patient states and
state transitions.
( 2
min )
Exfiltration of data via email is a serious cybersecurity threat for many
organizations. Detecting data exfiltration (anomaly) patterns typically
requires labeling, most often done by a human annotator, to reduce the high
number of false alarms. Active Learning (AL) is a promising approach for
labeling data efficiently, but it needs to choose an efficient order in which
cases are to be labeled, and there are uncertainties as to what scoring
procedure should be used to prioritize cases for labeling, especially when
detecting rare cases of interest is crucial. We propose an adaptive AL sampling
strategy that leverages the underlying prior data distribution, as well as
model uncertainty, to produce batches of cases to be labeled that contain
instances of rare anomalies. We show that (1) the classifier benefits from a
batch of representative and informative instances of both normal and anomalous
examples, (2) unsupervised anomaly detection plays a useful role in building
the classifier in the early stages of training when relatively little labeling
has been done thus far. Our approach to AL for anomaly detection outperformed
existing AL approaches on three highly unbalanced UCI benchmarks and on one
real-world redacted email data set.
( 2
min )
This report presents the technical details of our submission on the EGO4D
Audio-Visual (AV) Automatic Speech Recognition Challenge 2023 from the
OxfordVGG team. We present WhisperX, a system for efficient speech
transcription of long-form audio with word-level time alignment, along with two
text normalisers which are publicly available. Our final submission obtained
56.0% of the Word Error Rate (WER) on the challenge test set, ranked 1st on the
leaderboard. All baseline codes and models are available on
https://github.com/m-bain/whisperX.
( 2
min )
Recent regulation on right-to-be-forgotten emerges tons of interest in
unlearning pre-trained machine learning models. While approximating a
straightforward yet expensive approach of retrain-from-scratch, recent machine
unlearning methods unlearn a sample by updating weights to remove its influence
on the weight parameters. In this paper, we introduce a simple yet effective
approach to remove a data influence on the deep generative model. Inspired by
works in multi-task learning, we propose to manipulate gradients to regularize
the interplay of influence among samples by projecting gradients onto the
normal plane of the gradients to be retained. Our work is agnostic to
statistics of the removal samples, outperforming existing baselines while
providing theoretical analysis for the first time in unlearning a generative
model.
( 2
min )
In this paper we present a new classification method based on Dictionary
Learning (DL). The main contribution consists of a kernel version of incoherent
DL, derived from its standard linear counterpart. We also propose an
improvement of the AK-SVD algorithm concerning the representation update. Our
algorithms are tested on several popular databases of classification problems.
( 2
min )
In this letter, we propose the use of a meta-learning based precoder
optimization framework to directly optimize the Rate-Splitting Multiple Access
(RSMA) precoders with partial Channel State Information at the Transmitter
(CSIT). By exploiting the overfitting of the compact neural network to maximize
the explicit Average Sum-Rate (ASR) expression, we effectively bypass the need
for any other training data while minimizing the total running time. Numerical
results reveal that the meta-learning based solution achieves similar ASR
performance to conventional precoder optimization in medium-scale scenarios,
and significantly outperforms sub-optimal low complexity precoder algorithms in
the large-scale regime.
( 2
min )
When to solve math problems, most language models take a sampling strategy to
predict next word according conditional probabilities. In the math reasoning
step, it may generate wrong answer. Considering math problems are
deterministic, we propose a mixed policy exploration approach to solve math
problems with reinforcement learning. In peculiar, we propose a two level token
exploration policy: the abstract level explores next token with probability and
the second level is deterministic. Specifically, the abstract level policy will
decide whether the token is operator or operand with probability sampling,
while the second level is deterministic to select next token with the highest
score in a greedy way. We test our method on GSM8K dataset with GPT-2 model,
and demonstrate more than $2\%$ performance gain. Our implementation is
available at https://github.com/vividitytech/math_lm_rl.
( 2
min )
We study the generalization properties of batched predictors, i.e., models
tasked with predicting the mean label of a small set (or batch) of examples.
The batched prediction paradigm is particularly relevant for models deployed to
determine the quality of a group of compounds in preparation for offline
testing. By utilizing a suitable generalization of the Rademacher complexity,
we prove that batched predictors come with exponentially stronger
generalization guarantees as compared to the standard per-sample approach.
Surprisingly, the proposed bound holds independently of overparametrization.
Our theoretical insights are validated experimentally for various tasks,
architectures, and applications.
( 2
min )
High-dimensional clinical data have become invaluable resources for genetic
studies, due to their accessibility in biobank-scale datasets and the
development of high performance modeling techniques especially using deep
learning. Recent work has shown that low dimensional embeddings of these
clinical data learned by variational autoencoders (VAE) can be used for
genome-wide association studies and polygenic risk prediction. In this work, we
consider multiple unsupervised learning methods for learning disentangled
representations, namely autoencoders, VAE, beta-VAE, and FactorVAE, in the
context of genetic association studies. Using spirograms from UK Biobank as a
running example, we observed improvements in the number of genome-wide
significant loci, heritability, and performance of polygenic risk scores for
asthma and chronic obstructive pulmonary disease by using FactorVAE or
beta-VAE, compared to standard VAE or non-variational autoencoders. FactorVAEs
performed effectively across multiple values of the regularization
hyperparameter, while beta-VAEs were much more sensitive to the hyperparameter
values.
( 2
min )
Amazon Lex is a service that allows you to quickly and easily build conversational bots (“chatbots”), virtual agents, and interactive voice response (IVR) systems for applications such as Amazon Connect. Artificial intelligence (AI) and machine learning (ML) have been a focus for Amazon for over 20 years, and many of the capabilities that customers use […]
( 10
min )
In today’s digital world, most consumers would rather find answers to their customer service questions on their own rather than taking the time to reach out to businesses and/or service providers. This blog post explores an innovative solution to build a question and answer chatbot in Amazon Lex that uses existing FAQs from your website. […]
( 9
min )
Spam emails, also known as junk mail, are sent to a large number of users at once and often contain scams, phishing content, or cryptic messages. Spam emails are sometimes sent manually by a human, but most often they are sent using a bot. Examples of spam emails include fake ads, chain emails, and impersonation […]
( 6
min )
Today, we are excited to announce that Llama 2 foundation models developed by Meta are available for customers through Amazon SageMaker JumpStart. The Llama 2 family of large language models (LLMs) is a collection of pre-trained and fine-tuned generative text models ranging in scale from 7 billion to 70 billion parameters. Fine-tuned LLMs, called Llama-2-chat, […]
( 14
min )
The “Portal: Prelude RTX” gaming mod — a remastering of the popular unofficial “Portal” prequel — comes with full ray tracing, DLSS 3 and RTX IO technology for cutting-edge, AI-powered graphics that rejuvenate the legendary mod for gamers, creators, developers and others to experience it anew.
( 7
min )
A new $5+ million partnership aims to explore ways the development of artificial intelligence (AI) can support a thriving, innovative local news field, and ensure local news organizations shape the future of this emerging technology.
( 3
min )
EECS professor appointed to new professorship in the MIT Schwarzman College of Computing.
( 6
min )
With cloud computing, as compute power and data became more available, machine learning (ML) is now making an impact across every industry and is a core part of every business and industry. Amazon SageMaker Studio is the first fully integrated ML development environment (IDE) with a web-based visual interface. You can perform all ML development […]
( 10
min )
There is a recent interview, The Ethical Puzzle of Sentient AI, where a professor said, “But there’s also the problem that I’ve called the ‘gaming problem’ — that when the system has access to trillions of words of training data, and has been trained with the goal of mimicking human behavior, the sorts of behavior patterns… Read More »LLMs: Does human text data make generative AI an entity?
The post LLMs: Does human text data make generative AI an entity? appeared first on Data Science Central.
( 19
min )
The generalization performance of deep neural networks with regard to the
optimization algorithm is one of the major concerns in machine learning. This
performance can be affected by various factors. In this paper, we theoretically
prove that the Lipschitz constant of a loss function is an important factor to
diminish the generalization error of the output model obtained by Adam or
AdamW. The results can be used as a guideline for choosing the loss function
when the optimization algorithm is Adam or AdamW. In addition, to evaluate the
theoretical bound in a practical setting, we choose the human age estimation
problem in computer vision. For assessing the generalization better, the
training and test datasets are drawn from different distributions. Our
experimental evaluation shows that the loss function with a lower Lipschitz
constant and maximum value improves the generalization of the model trained by
Adam or AdamW.
( 2
min )
Despite the dominance and effectiveness of scaling, resulting in large
networks with hundreds of billions of parameters, the necessity to train
overparametrized models remains poorly understood, and alternative approaches
do not necessarily make it cheaper to train high-performance models. In this
paper, we explore low-rank training techniques as an alternative approach to
training large neural networks. We introduce a novel method called ReLoRA,
which utilizes low-rank updates to train high-rank networks. We apply ReLoRA to
pre-training transformer language models with up to 350M parameters and
demonstrate comparable performance to regular neural network training.
Furthermore, we observe that the efficiency of ReLoRA increases with model
size, making it a promising approach for training multi-billion-parameter
networks efficiently. Our findings shed light on the potential of low-rank
training techniques and their implications for scaling laws.
( 2
min )
We present APAC-Net, an alternating population and agent control neural
network for solving stochastic mean field games (MFGs). Our algorithm is geared
toward high-dimensional instances of MFGs that are beyond reach with existing
solution methods. We achieve this in two steps. First, we take advantage of the
underlying variational primal-dual structure that MFGs exhibit and phrase it as
a convex-concave saddle point problem. Second, we parameterize the value and
density functions by two neural networks, respectively. By phrasing the problem
in this manner, solving the MFG can be interpreted as a special case of
training a generative adversarial network (GAN). We show the potential of our
method on up to 100-dimensional MFG problems.
( 2
min )
Federated learning (FL) has evolved as a prominent method for edge devices to
cooperatively create a unified prediction model while securing their sensitive
training data local to the device. Despite the existence of numerous research
frameworks for simulating FL algorithms, they do not facilitate comprehensive
deployment for automatic speech recognition tasks on heterogeneous edge
devices. This is where Ed-Fed, a comprehensive and generic FL framework, comes
in as a foundation for future practical FL system research. We also propose a
novel resource-aware client selection algorithm to optimise the waiting time in
the FL settings. We show that our approach can handle the straggler devices and
dynamically set the training time for the selected devices in a round. Our
evaluation has shown that the proposed approach significantly optimises waiting
time in FL compared to conventional random client selection methods.
( 2
min )
The current cut selection algorithm used in mixed-integer programming solvers
has remained largely unchanged since its creation. In this paper, we propose a
set of new cut scoring measures, cut filtering techniques, and stopping
criteria, extending the current state-of-the-art algorithm and obtaining a 4\%
performance improvement for SCIP over the MIPLIB 2017 benchmark set.
( 2
min )
Controlling nonlinear dynamical systems using machine learning allows to not
only drive systems into simple behavior like periodicity but also to more
complex arbitrary dynamics. For this, it is crucial that a machine learning
system can be trained to reproduce the target dynamics sufficiently well. On
the example of forcing a chaotic parametrization of the Lorenz system into
intermittent dynamics, we show first that classical reservoir computing excels
at this task. In a next step, we compare those results based on different
amounts of training data to an alternative setup, where next-generation
reservoir computing is used instead. It turns out that while delivering
comparable performance for usual amounts of training data, next-generation RC
significantly outperforms in situations where only very limited data is
available. This opens even further practical control applications in real world
problems where data is restricted.
( 2
min )
Recent advances in large language models have led to renewed interest in
natural language processing in healthcare using the free text of clinical
notes. One distinguishing characteristic of clinical notes is their long time
span over multiple long documents. The unique structure of clinical notes
creates a new design choice: when the context length for a language model
predictor is limited, which part of clinical notes should we choose as the
input? Existing studies either choose the inputs with domain knowledge or
simply truncate them. We propose a framework to analyze the sections with high
predictive power. Using MIMIC-III, we show that: 1) predictive power
distribution is different between nursing notes and discharge notes and 2)
combining different types of notes could improve performance when the context
length is large. Our findings suggest that a carefully selected sampling
function could enable more efficient information extraction from clinical
notes.
( 2
min )
We propose a novel task-agnostic in-domain pre-training method that sits
between generic pre-training and fine-tuning. Our approach selectively masks
in-domain keywords, i.e., words that provide a compact representation of the
target domain. We identify such keywords using KeyBERT (Grootendorst, 2020). We
evaluate our approach using six different settings: three datasets combined
with two distinct pre-trained language models (PLMs). Our results reveal that
the fine-tuned PLMs adapted using our in-domain pre-training strategy
outperform PLMs that used in-domain pre-training with random masking as well as
those that followed the common pre-train-then-fine-tune paradigm. Further, the
overhead of identifying in-domain keywords is reasonable, e.g., 7-15% of the
pre-training time (for two epochs) for BERT Large (Devlin et al., 2019).
( 2
min )
Understanding how the statistical and geometric properties of neural activity
relate to performance is a key problem in theoretical neuroscience and deep
learning. Here, we calculate how correlations between object representations
affect the capacity, a measure of linear separability. We show that for
spherical object manifolds, introducing correlations between centroids
effectively pushes the spheres closer together, while introducing correlations
between the axes effectively shrinks their radii, revealing a duality between
correlations and geometry with respect to the problem of classification. We
then apply our results to accurately estimate the capacity of deep network
data.
( 2
min )
We analyze statistical discrimination in hiring markets using a multi-armed
bandit model. Myopic firms face workers arriving with heterogeneous observable
characteristics. The association between the worker's skill and characteristics
is unknown ex ante; thus, firms need to learn it. Laissez-faire causes
perpetual underestimation: minority workers are rarely hired, and therefore,
the underestimation tends to persist. Even a marginal imbalance in the
population ratio frequently results in perpetual underestimation. We propose
two policy solutions: a novel subsidy rule (the hybrid mechanism) and the
Rooney Rule. Our results indicate that temporary affirmative actions
effectively alleviate discrimination stemming from insufficient data.
( 2
min )
Hypothesis transfer learning (HTL) contrasts domain adaptation by allowing
for a previous task leverage, named the source, into a new one, the target,
without requiring access to the source data. Indeed, HTL relies only on a
hypothesis learnt from such source data, relieving the hurdle of expansive data
storage and providing great practical benefits. Hence, HTL is highly beneficial
for real-world applications relying on big data. The analysis of such a method
from a theoretical perspective faces multiple challenges, particularly in
classification tasks. This paper deals with this problem by studying the
learning theory of HTL through algorithmic stability, an attractive theoretical
framework for machine learning algorithms analysis. In particular, we are
interested in the statistical behaviour of the regularized empirical risk
minimizers in the case of binary classification. Our stability analysis
provides learning guarantees under mild assumptions. Consequently, we derive
several complexity-free generalization bounds for essential statistical
quantities like the training error, the excess risk and cross-validation
estimates. These refined bounds allow understanding the benefits of transfer
learning and comparing the behaviour of standard losses in different scenarios,
leading to valuable insights for practitioners.
( 2
min )
Understanding the implicit regularization imposed by neural network
architectures and gradient based optimization methods is a key challenge in
deep learning and AI. In this work we provide sharp results for the implicit
regularization imposed by the gradient flow of Diagonal Linear Networks (DLNs)
in the over-parameterized regression setting and, potentially surprisingly,
link this to the phenomenon of phase transitions in generalized hardness of
approximation (GHA). GHA generalizes the phenomenon of hardness of
approximation from computer science to, among others, continuous and robust
optimization. It is well-known that the $\ell^1$-norm of the gradient flow of
DLNs with tiny initialization converges to the objective function of basis
pursuit. We improve upon these results by showing that the gradient flow of
DLNs with tiny initialization approximates minimizers of the basis pursuit
optimization problem (as opposed to just the objective function), and we obtain
new and sharp convergence bounds w.r.t.\ the initialization size. Non-sharpness
of our results would imply that the GHA phenomenon would not occur for the
basis pursuit optimization problem -- which is a contradiction -- thus implying
sharpness. Moreover, we characterize $\textit{which}$ $\ell_1$ minimizer of the
basis pursuit problem is chosen by the gradient flow whenever the minimizer is
not unique. Interestingly, this depends on the depth of the DLN.
( 3
min )
We present APAC-Net, an alternating population and agent control neural
network for solving stochastic mean field games (MFGs). Our algorithm is geared
toward high-dimensional instances of MFGs that are beyond reach with existing
solution methods. We achieve this in two steps. First, we take advantage of the
underlying variational primal-dual structure that MFGs exhibit and phrase it as
a convex-concave saddle point problem. Second, we parameterize the value and
density functions by two neural networks, respectively. By phrasing the problem
in this manner, solving the MFG can be interpreted as a special case of
training a generative adversarial network (GAN). We show the potential of our
method on up to 100-dimensional MFG problems.
( 2
min )
In this paper, we investigate the impact of numerical instability on the
reliability of sampling, density evaluation, and evidence lower bound (ELBO)
estimation in variational flows. We first empirically demonstrate that common
flows can exhibit a catastrophic accumulation of error: the numerical flow map
deviates significantly from the exact map -- which affects sampling -- and the
numerical inverse flow map does not accurately recover the initial input --
which affects density and ELBO computations. Surprisingly though, we find that
results produced by flows are often accurate enough for applications despite
the presence of serious numerical instability. In this work, we treat
variational flows as dynamical systems, and leverage shadowing theory to
elucidate this behavior via theoretical guarantees on the error of sampling,
density evaluation, and ELBO estimation. Finally, we develop and empirically
test a diagnostic procedure that can be used to validate results produced by
numerically unstable flows in practice.
( 2
min )
Large-language models (LLMs) such as GPT-4 caught the interest of many
scientists. Recent studies suggested that these models could be useful in
chemistry and materials science. To explore these possibilities, we organized a
hackathon.
This article chronicles the projects built as part of this hackathon.
Participants employed LLMs for various applications, including predicting
properties of molecules and materials, designing novel interfaces for tools,
extracting knowledge from unstructured data, and developing new educational
applications.
The diverse topics and the fact that working prototypes could be generated in
less than two days highlight that LLMs will profoundly impact the future of our
fields. The rich collection of ideas and projects also indicates that the
applications of LLMs are not limited to materials science and chemistry but
offer potential benefits to a wide range of scientific disciplines.
( 3
min )
PIGINet leverages machine learning to streamline and enhance household robots' task and motion planning, by assessing and filtering feasible solutions in complex environments.
( 9
min )
A new report by MIT researchers highlights the potential of generative AI to help workers with certain writing assignments.
( 9
min )
Researchers create a privacy technique that protects sensitive data while maintaining a machine-learning model’s performance.
( 10
min )
We study the adaption of soft actor-critic (SAC) from continuous action space
to discrete action space. We revisit vanilla SAC and provide an in-depth
understanding of its Q value underestimation and performance instability issues
when applied to discrete settings. We thereby propose entropy-penalty and
double average Q-learning with Q-clip to address these issues. Extensive
experiments on typical benchmarks with discrete action space, including Atari
games and a large-scale MOBA game, show the efficacy of our proposed method.
Our code is at:https://github.com/coldsummerday/Revisiting-Discrete-SAC.
( 2
min )
The coupling of deep reinforcement learning to numerical flow control
problems has recently received a considerable attention, leading to
groundbreaking results and opening new perspectives for the domain. Due to the
usually high computational cost of fluid dynamics solvers, the use of parallel
environments during the learning process represents an essential ingredient to
attain efficient control in a reasonable time. Yet, most of the deep
reinforcement learning literature for flow control relies on on-policy
algorithms, for which the massively parallel transition collection may break
theoretical assumptions and lead to suboptimal control models. To overcome this
issue, we propose a parallelism pattern relying on partial-trajectory buffers
terminated by a return bootstrapping step, allowing a flexible use of parallel
environments while preserving the on-policiness of the updates. This approach
is illustrated on a CPU-intensive continuous flow control problem from the
literature.
( 2
min )
When measuring rare processes at Belle II, a huge luminosity is required,
which means a large number of simulations are necessary to determine signal
efficiencies and background contributions. However, this process demands high
computation costs while most of the simulated data, in particular in case of
background, are discarded by the event selection. Thus, filters using graph
neural networks are introduced at an early stage to save the resources for the
detector simulation and reconstruction of events discarded at analysis level.
In our work, we improved the performance of the filters using graph attention
and investigated statistical methods including sampling and reweighting to deal
with the biases introduced by the filtering.
( 2
min )
For prediction of clustered time-to-event data, we propose a new deep neural
network based gamma frailty model (DNN-FM). An advantage of the proposed model
is that the joint maximization of the new h-likelihood provides maximum
likelihood estimators for fixed parameters and best unbiased predictors for
random frailties. Thus, the proposed DNN-FM is trained by using a negative
profiled h-likelihood as a loss function, constructed by profiling out the
non-parametric baseline hazard. Experimental studies show that the proposed
method enhances the prediction performance of the existing methods. A real data
analysis shows that the inclusion of subject-specific frailties helps to
improve prediction of the DNN based Cox model (DNN-Cox).
( 2
min )
The computation necessary for training Transformer-based language models has
skyrocketed in recent years. This trend has motivated research on efficient
training algorithms designed to improve training, validation, and downstream
performance faster than standard training. In this work, we revisit three
categories of such algorithms: dynamic architectures (layer stacking, layer
dropping), batch selection (selective backprop, RHO loss), and efficient
optimizers (Lion, Sophia). When pre-training BERT and T5 with a fixed
computation budget using such methods, we find that their training, validation,
and downstream gains vanish compared to a baseline with a fully-decayed
learning rate. We define an evaluation protocol that enables computation to be
done on arbitrary machines by mapping all computation time to a reference
machine which we call reference system time. We discuss the limitations of our
proposed protocol and release our code to encourage rigorous research in
efficient training procedures: https://github.com/JeanKaddour/NoTrainNoGain.
( 2
min )
Recently Chen and Poor initiated the study of learning mixtures of linear
dynamical systems. While linear dynamical systems already have wide-ranging
applications in modeling time-series data, using mixture models can lead to a
better fit or even a richer understanding of underlying subpopulations
represented in the data. In this work we give a new approach to learning
mixtures of linear dynamical systems that is based on tensor decompositions. As
a result, our algorithm succeeds without strong separation conditions on the
components, and can be used to compete with the Bayes optimal clustering of the
trajectories. Moreover our algorithm works in the challenging
partially-observed setting. Our starting point is the simple but powerful
observation that the classic Ho-Kalman algorithm is a close relative of modern
tensor decomposition methods for learning latent variable models. This gives us
a playbook for how to extend it to work with more complicated generative
models.
( 2
min )
For prediction of clustered time-to-event data, we propose a new deep neural
network based gamma frailty model (DNN-FM). An advantage of the proposed model
is that the joint maximization of the new h-likelihood provides maximum
likelihood estimators for fixed parameters and best unbiased predictors for
random frailties. Thus, the proposed DNN-FM is trained by using a negative
profiled h-likelihood as a loss function, constructed by profiling out the
non-parametric baseline hazard. Experimental studies show that the proposed
method enhances the prediction performance of the existing methods. A real data
analysis shows that the inclusion of subject-specific frailties helps to
improve prediction of the DNN based Cox model (DNN-Cox).
( 2
min )
We investigate a framework for binary image denoising via restricted
Boltzmann machines (RBMs) that introduces a denoising objective in quadratic
unconstrained binary optimization (QUBO) form and is well-suited for quantum
annealing. The denoising objective is attained by balancing the distribution
learned by a trained RBM with a penalty term for derivations from the noisy
image. We derive the statistically optimal choice of the penalty parameter
assuming the target distribution has been well-approximated, and further
suggest an empirically supported modification to make the method robust to that
idealistic assumption. We also show under additional assumptions that the
denoised images attained by our method are, in expectation, strictly closer to
the noise-free images than the noisy images are. While we frame the model as an
image denoising model, it can be applied to any binary data. As the QUBO
formulation is well-suited for implementation on quantum annealers, we test the
model on a D-Wave Advantage machine, and also test on data too large for
current quantum annealers by approximating QUBO solutions through classical
heuristics.
( 2
min )
We consider stochastic optimization problems where data is drawn from a
Markov chain. Existing methods for this setting crucially rely on knowing the
mixing time of the chain, which in real-world applications is usually unknown.
We propose the first optimization method that does not require the knowledge of
the mixing time, yet obtains the optimal asymptotic convergence rate when
applied to convex problems. We further show that our approach can be extended
to: (i) finding stationary points in non-convex optimization with Markovian
data, and (ii) obtaining better dependence on the mixing time in temporal
difference (TD) learning; in both cases, our method is completely oblivious to
the mixing time. Our method relies on a novel combination of multi-level Monte
Carlo (MLMC) gradient estimation together with an adaptive learning method.
( 2
min )
We propose a goodness-of-fit measure for probability densities modeling
observations with varying dimensionality, such as text documents of differing
lengths or variable-length sequences. The proposed measure is an instance of
the kernel Stein discrepancy (KSD), which has been used to construct
goodness-of-fit tests for unnormalized densities. The KSD is defined by its
Stein operator: current operators used in testing apply to fixed-dimensional
spaces. As our main contribution, we extend the KSD to the variable-dimension
setting by identifying appropriate Stein operators, and propose a novel KSD
goodness-of-fit test. As with the previous variants, the proposed KSD does not
require the density to be normalized, allowing the evaluation of a large class
of models. Our test is shown to perform well in practice on discrete sequential
data benchmarks.
( 2
min )
Recently Chen and Poor initiated the study of learning mixtures of linear
dynamical systems. While linear dynamical systems already have wide-ranging
applications in modeling time-series data, using mixture models can lead to a
better fit or even a richer understanding of underlying subpopulations
represented in the data. In this work we give a new approach to learning
mixtures of linear dynamical systems that is based on tensor decompositions. As
a result, our algorithm succeeds without strong separation conditions on the
components, and can be used to compete with the Bayes optimal clustering of the
trajectories. Moreover our algorithm works in the challenging
partially-observed setting. Our starting point is the simple but powerful
observation that the classic Ho-Kalman algorithm is a close relative of modern
tensor decomposition methods for learning latent variable models. This gives us
a playbook for how to extend it to work with more complicated generative
models.
( 2
min )
Stochastic Gradient Descent (SGD) is one of the simplest and most popular
algorithms in modern statistical and machine learning due to its computational
and memory efficiency. Various averaging schemes have been proposed to
accelerate the convergence of SGD in different settings. In this paper, we
explore a general averaging scheme for SGD. Specifically, we establish the
asymptotic normality of a broad range of weighted averaged SGD solutions and
provide asymptotically valid online inference approaches. Furthermore, we
propose an adaptive averaging scheme that exhibits both optimal statistical
rate and favorable non-asymptotic convergence, drawing insights from the
optimal weight for the linear model in terms of non-asymptotic mean squared
error (MSE).
( 2
min )
Image Source: Author Introduction Data Engineers and Data Scientists need data for their Day-to-Day job. Of course, It could be for Data Analytics, Data Prediction, Data Mining, Building Machine Learning Models Etc., All these are taken care of by the respective team members and they need to work towards identifying relevant data sources, and associated with… Read More »A Detailed Guide for Data Handling Techniques in Data Science
The post A Detailed Guide for Data Handling Techniques in Data Science appeared first on Data Science Central.
( 26
min )
Recent years have shown amazing growth in deep learning neural networks (DNNs). This growth can be seen in more accurate models and even opening new possibilities with generative AI: large language models (LLMs) that synthesize natural language, text-to-image generators, and more. These increased capabilities of DNNs come with the cost of having massive models that […]
( 11
min )
A watershed moment on Nov. 22, 2022, was mostly virtual, yet it shook the foundations of nearly every industry on the planet. On that day, OpenAI released ChatGPT, the most advanced artificial intelligence chatbot ever developed. This set off demand for generative AI applications that help businesses become more efficient, from providing consumers with answers Read article >
( 11
min )
Arise, members! Capcom’s legendary role-playing game Dragon’s Dogma: Dark Arisen joins the GeForce NOW library today. The RPG and THQ Nordic’s Jagged Alliance 3 are newly supported on GeForce NOW, playable on nearly any device. From Dusk Till Pawn Become the Arisen and take up the challenge in Capcom’s critically acclaimed RPG. Set in a Read article >
( 5
min )
Because headphones rank among the most popular wearables in the market, we have an exciting opportunity to expand their capabilities through integrating existing sensors with supplementary ones to enable a wide variety of experiences that go beyond traditional audio control.
The post Thinking beyond audio: Augmenting headphones for everyday digital interactions appeared first on Microsoft Research.
( 12
min )
A crack NVIDIA team of five machine learning experts spread across four continents won all three tasks in a hotly contested, prestigious competition to build state-of-the-art recommendation systems. The results reflect the group’s savvy applying the NVIDIA AI platform to real-world challenges for these engines of the digital economy. Recommenders serve up trillions of search Read article >
( 6
min )
Startup MosaicML is on a mission to help the AI community improve prediction accuracy, decrease costs and save time by providing tools for easy training and deployment of large AI models. In this episode of NVIDIA’s AI Podcast, host Noah Kravitz speaks with MosaicML CEO and co-founder Naveen Rao about how the company aims to Read article >
( 5
min )
Differences in gait patterns of children with Duchenne muscular dystrophy
(DMD) and typically-developing (TD) peers are visible to the eye, but
quantifications of those differences outside of the gait laboratory have been
elusive. In this work, we measured vertical, mediolateral, and anteroposterior
acceleration using a waist-worn iPhone accelerometer during ambulation across a
typical range of velocities. Fifteen TD and fifteen DMD children from 3-16
years of age underwent eight walking/running activities, including five 25
meters walk/run speed-calibration tests at a slow walk to running speeds (SC-L1
to SC-L5), a 6-minute walk test (6MWT), a 100 meters fast-walk/jog/run
(100MRW), and a free walk (FW). For clinical anchoring purposes, participants
completed a Northstar Ambulatory Assessment (NSAA). We extracted temporospatial
gait clinical features (CFs) and applied multiple machine learning (ML)
approaches to differentiate between DMD and TD children using extracted
temporospatial gait CFs and raw data. Extracted temporospatial gait CFs showed
reduced step length and a greater mediolateral component of total power (TP)
consistent with shorter strides and Trendelenberg-like gait commonly observed
in DMD. ML approaches using temporospatial gait CFs and raw data varied in
effectiveness at differentiating between DMD and TD controls at different
speeds, with an accuracy of up to 100%. We demonstrate that by using ML with
accelerometer data from a consumer-grade smartphone, we can capture
DMD-associated gait characteristics in toddlers to teens.
( 3
min )
Recent groundbreaking developments on generative modeling have sparked
interest in practical single-model attribution. Such methods predict whether a
sample was generated by a specific generator or not, for instance, to prove
intellectual property theft. However, previous works are either limited to the
closed-world setting or require undesirable changes of the generative model. We
address these shortcomings by proposing FLIPAD, a new approach for single-model
attribution in the open-world setting based on final-layer inversion and
anomaly detection. We show that the utilized final-layer inversion can be
reduced to a convex lasso optimization problem, making our approach
theoretically sound and computationally efficient. The theoretical findings are
accompanied by an experimental study demonstrating the effectiveness of our
approach, outperforming the existing methods.
( 2
min )
We propose a simple and efficient approach to generate a prediction intervals
(PI) for approximated and forecasted trends. Our method leverages a weighted
asymmetric loss function to estimate the lower and upper bounds of the PI, with
the weights determined by its coverage probability. We provide a concise
mathematical proof of the method, show how it can be extended to derive PIs for
parametrised functions and argue why the method works for predicting PIs of
dependent variables. The presented tests of the method on a real-world
forecasting task using a neural network-based model show that it can produce
reliable PIs in complex machine learning scenarios.
( 2
min )
The importance of high data quality is increasing with the growing impact and
distribution of ML systems and big data. Also the planned AI Act from the
European commission defines challenging legal requirements for data quality
especially for the market introduction of safety relevant ML systems. In this
paper we introduce a novel approach that supports the data quality assurance
process of multiple data quality aspects. This approach enables the
verification of quantitative data quality requirements. The concept and
benefits are introduced and explained on small example data sets. How the
method is applied is demonstrated on the well known MNIST data set based an
handwritten digits.
( 2
min )
Deep generative chemistry models emerge as powerful tools to expedite drug
discovery. However, the immense size and complexity of the structural space of
all possible drug-like molecules pose significant obstacles, which could be
overcome with hybrid architectures combining quantum computers with deep
classical networks. As the first step toward this goal, we built a compact
discrete variational autoencoder (DVAE) with a Restricted Boltzmann Machine
(RBM) of reduced size in its latent layer. The size of the proposed model was
small enough to fit on a state-of-the-art D-Wave quantum annealer and allowed
training on a subset of the ChEMBL dataset of biologically active compounds.
Finally, we generated 2331 novel chemical structures with medicinal chemistry
and synthetic accessibility properties in the ranges typical for molecules from
ChEMBL. The presented results demonstrate the feasibility of using already
existing or soon-to-be-available quantum computing devices as testbeds for
future drug discovery applications.
( 2
min )
Partial differential equations (PDEs) are a model candidate for soft sensors
in industrial processes with spatiotemporal dependence. Although
physics-informed neural networks (PINNs) are a promising machine learning
method for solving PDEs, they are infeasible for the nonhomogeneous PDEs with
unmeasurable source terms. To this end, a coupled PINN (CPINN) with a recurrent
prediction (RP) learning strategy (CPINN- RP) is proposed. First, CPINN
composed of NetU and NetG is proposed. NetU is for approximating PDEs solutions
and NetG is for regularizing the training of NetU. The two networks are
integrated into a data-physics-hybrid loss function. Then, we theoretically
prove that the proposed CPINN has a satisfying approximation capability for
solutions to nonhomogeneous PDEs with unmeasurable source terms. Besides the
theoretical aspects, we propose a hierarchical training strategy to optimize
and couple NetU and NetG. Secondly, NetU-RP is proposed for compensating
information loss in data sampling to improve the prediction performance, in
which RP is the recurrently delayed outputs of well-trained CPINN and hard
sensors. Finally, the artificial and practical datasets are used to verify the
feasibility and effectiveness of CPINN-RP for soft sensors.
( 3
min )
This paper introduces Local Learner (2L), an algorithm for providing a set of
reference strategies to guide the search for programmatic strategies in
two-player zero-sum games. Previous learning algorithms, such as Iterated Best
Response (IBR), Fictitious Play (FP), and Double-Oracle (DO), can be
computationally expensive or miss important information for guiding search
algorithms. 2L actively selects a set of reference strategies to improve the
search signal. We empirically demonstrate the advantages of our approach while
guiding a local search algorithm for synthesizing strategies in three games,
including MicroRTS, a challenging real-time strategy game. Results show that 2L
learns reference strategies that provide a stronger search signal than IBR, FP,
and DO. We also simulate a tournament of MicroRTS, where a synthesizer using 2L
outperformed the winners of the two latest MicroRTS competitions, which were
programmatic strategies written by human programmers.
( 2
min )
We propose a new online algorithm for cumulative regret minimization in a
stochastic linear bandit. The algorithm pulls the arm with the highest
estimated reward in a linear model trained on its perturbed history. Therefore,
we call it perturbed-history exploration in a linear bandit (LinPHE). The
perturbed history is a mixture of observed rewards and randomly generated
i.i.d. pseudo-rewards. We derive a $\tilde{O}(d \sqrt{n})$ gap-free bound on
the $n$-round regret of LinPHE, where $d$ is the number of features. The key
steps in our analysis are new concentration and anti-concentration bounds on
the weighted sum of Bernoulli random variables. To show the generality of our
design, we generalize LinPHE to a logistic model. We evaluate our algorithms
empirically and show that they are practical.
( 2
min )
Recent work has applied supervised deep learning to derive continuous
symmetry transformations that preserve the data labels and to obtain the
corresponding algebras of symmetry generators. This letter introduces two
improved algorithms that significantly speed up the discovery of these symmetry
transformations. The new methods are demonstrated by deriving the complete set
of generators for the unitary groups U(n) and the exceptional Lie groups $G_2$,
$F_4$, and $E_6$. A third post-processing algorithm renders the found
generators in sparse form. We benchmark the performance improvement of the new
algorithms relative to the standard approach. Given the significant complexity
of the exceptional Lie groups, our results demonstrate that this
machine-learning method for discovering symmetries is completely general and
can be applied to a wide variety of labeled datasets.
( 2
min )
A novel human emotion recognition method based on automatically selected
Galvanic Skin Response (GSR) signal features and SVM is proposed in this paper.
GSR signals were acquired by e-Health Sensor Platform V2.0. Then, the data is
de-noised by wavelet function and normalized to get rid of the individual
difference. 30 features are extracted from the normalized data, however,
directly using of these features will lead to a low recognition rate. In order
to gain the optimized features, a covariance based feature selection is
employed in our method. Finally, a SVM with input of the optimized features is
utilized to achieve the human emotion recognition. The experimental results
indicate that the proposed method leads to good human emotion recognition, and
the recognition accuracy is more than 66.67%.
( 2
min )
Score based generative models are a new class of generative models that have
been shown to accurately generate high dimensional calorimeter datasets. Recent
advances in generative models have used images with 3D voxels to represent and
model complex calorimeter showers. Point clouds, however, are likely a more
natural representation of calorimeter showers, particularly in calorimeters
with high granularity. Point clouds preserve all of the information of the
original simulation, more naturally deal with sparse datasets, and can be
implemented with more compact models and data files. In this work, two
state-of-the-art score based models are trained on the same set of calorimeter
simulation and directly compared.
( 2
min )
Business statistics play a crucial role in implementing a data-driven
strategic plan at the enterprise level to employ various analytics where the
outcomes of such a plan enable an enterprise to enhance the decision-making
process or to mitigate risks to the organization. In this work, a strategic
plan informed by the statistical analysis is introduced for a financial company
called LendingClub, where the plan is comprised of exploring the possibility of
onboarding a big data platform along with advanced feature selection
capacities. The main objectives of such a plan are to increase the company's
revenue while reducing the risks of granting loans to borrowers who cannot
return their loans. In this study, different hypotheses formulated to address
the company's concerns are studied, where the results reveal that the amount of
loans profoundly impacts the number of borrowers charging off their loans.
Also, the proposed strategic plan includes onboarding advanced analytics such
as machine learning technologies that allow the company to build better
generalized data-driven predictive models.
( 2
min )
Mental health disorders remain a significant challenge in modern healthcare,
with diagnosis and treatment often relying on subjective patient descriptions
and past medical history. To address this issue, we propose a personalized
mental health tracking and mood prediction system that utilizes patient
physiological data collected through personal health devices. Our system
leverages a decentralized learning mechanism that combines transfer and
federated machine learning concepts using smart contracts, allowing data to
remain on users' devices and enabling effective tracking of mental health
conditions for psychiatric treatment and management in a privacy-aware and
accountable manner. We evaluate our model using a popular mental health dataset
that demonstrates promising results. By utilizing connected health systems and
machine learning models, our approach offers a novel solution to the challenge
of providing psychiatrists with further insight into their patients' mental
health outside of traditional office visits.
( 2
min )
As more and more customers are looking to put machine learning (ML) workloads in production, there is a large push in organizations to shorten the development lifecycle of ML code. Many organizations prefer writing their ML code in a production-ready style in the form of Python methods and classes as opposed to an exploratory style […]
( 8
min )
Efficiency is vital in the face of escalating demand for cloud resources. And efficient power management strategies address the bottleneck of power availability in datacenters. Learn how we optimize power allocation to support sustainable resource usage.
The post Microsoft at ICALP 2023: Deploying cloud capacity robustly against power failures appeared first on Microsoft Research.
( 11
min )
Jacob Norris is a 3D artist and the president, co-founder and creative director of Sierra Division Studios — an outsource studio specializing in digital 3D content creation.
( 9
min )
Luca Carlone and Jonathan How of MIT LIDS discuss how future robots might perceive and interact with their environment.
( 8
min )
NVIDIA RTX is spinning new cycles for designs. Trek Bicycle is using GPUs to bring design concepts to life. The Wisconsin-based company, one of the largest bicycle manufacturers in the world, aims to create bikes with the highest-quality craftsmanship. With its new partner Lidl, an international retailer chain, Trek Bicycle also owns a cycling team, Read article >
( 7
min )
This research paper was accepted by 2023 USENIX Annual Technical Conference (ATC), which is dedicated to advancing the field of systems research. Whether they’re personal computers or cloud instances, it’s crucial to ensure that the computer systems people use every day are reliable and secure. The validity of these systems is critical because if storage […]
The post Renovating computer systems securely and progressively with APRON appeared first on Microsoft Research.
( 10
min )
When an enterprise project is low-profile (“below the radar”), then it is not likely to be the target of bad actors. Similarly, if some part of that project’s infrastructure fails or falters, then the consequences of the problem and/or the urgency of providing a solution are usually manageable. But when a high-profile (“above the radar”)… Read More »AIOps above the radar – Using AI to monitor your AI infrastructure
The post AIOps above the radar – Using AI to monitor your AI infrastructure appeared first on Data Science Central.
( 22
min )
Evolving technological advancements have created a far more data-centric world. This has dramatically changed the enterprise landscape, while also creating more data silos. The explosion of cybersecurity tools and mounds of data in modern enterprises have made it difficult to combine data to create a unified view. This has resulted in siloed data that’s also… Read More »Security data lakes and the future of organizational security
The post Security data lakes and the future of organizational security appeared first on Data Science Central.
( 21
min )
There is a recent article, Unraveling the Mystery of Human Consciousness, where it was stated that, “Consciousness makes us capable of experiencing the scent of a rose, the touch of a breeze, the taste of food, the sound of music, and the sight of a sunrise. We also have a unique ability to be aware… Read More »Sentience: AI has demystified human consciousness, intelligence
The post Sentience: AI has demystified human consciousness, intelligence appeared first on Data Science Central.
( 19
min )
Amazon Kendra and LLamaIndex can help with knowledge integration but fall short in connecting diverse knowledge sources, to enable efficient intelligent search. In this article, we compare the existing solutions and explain how to overcome their limitations using a Google Drive crawler. Companies often face difficulties in consolidating their knowledge base when their data is… Read More »Exploring intelligent search solutions: A comparative analysis of Amazon Kendra integration and large language model crawlers
The post Exploring intelligent search solutions: A comparative analysis of Amazon Kendra integration and large language model crawlers appeared first on Data Science Central.
( 27
min )
Identifying and suppressing unknown disturbances to dynamical systems is a
problem with applications in many different fields. In this Letter, we present
a model-free method to identify and suppress an unknown disturbance to an
unknown system based only on previous observations of the system under the
influence of a known forcing function. We find that, under very mild
restrictions on the training function, our method is able to robustly identify
and suppress a large class of unknown disturbances. We illustrate our scheme
with an example where a chaotic disturbance to the Lorenz system is identified
and suppressed.
( 2
min )
We discuss a vulnerability involving a category of attribution methods used
to provide explanations for the outputs of convolutional neural networks
working as classifiers. It is known that this type of networks are vulnerable
to adversarial attacks, in which imperceptible perturbations of the input may
alter the outputs of the model. In contrast, here we focus on effects that
small modifications in the model may cause on the attribution method without
altering the model outputs.
( 2
min )
We consider the problem of clustering privately a dataset in $\mathbb{R}^d$
that undergoes both insertion and deletion of points. Specifically, we give an
$\varepsilon$-differentially private clustering mechanism for the $k$-means
objective under continual observation. This is the first approximation
algorithm for that problem with an additive error that depends only
logarithmically in the number $T$ of updates. The multiplicative error is
almost the same as non privately. To do so we show how to perform dimension
reduction under continual observation and combine it with a differentially
private greedy approximation algorithm for $k$-means. We also partially extend
our results to the $k$-median problem.
( 2
min )
This research proposes a machine learning-based attack detection model for
power systems, specifically targeting smart grids. By utilizing data and logs
collected from Phasor Measuring Devices (PMUs), the model aims to learn system
behaviors and effectively identify potential security boundaries. The proposed
approach involves crucial stages including dataset pre-processing, feature
selection, model creation, and evaluation. To validate our approach, we used a
dataset used, consist of 15 separate datasets obtained from different PMUs,
relay snort alarms and logs. Three machine learning models: Random Forest,
Logistic Regression, and K-Nearest Neighbour were built and evaluated using
various performance metrics. The findings indicate that the Random Forest model
achieves the highest performance with an accuracy of 90.56% in detecting power
system disturbances and has the potential in assisting operators in
decision-making processes.
( 2
min )
Feature-distributed data, referred to data partitioned by features and stored
across multiple computing nodes, are increasingly common in applications with a
large number of features. This paper proposes a two-stage relaxed greedy
algorithm (TSRGA) for applying multivariate linear regression to such data. The
main advantage of TSRGA is that its communication complexity does not depend on
the feature dimension, making it highly scalable to very large data sets. In
addition, for multivariate response variables, TSRGA can be used to yield
low-rank coefficient estimates. The fast convergence of TSRGA is validated by
simulation experiments. Finally, we apply the proposed TSRGA in a financial
application that leverages unstructured data from the 10-K reports,
demonstrating its usefulness in applications with many dense large-dimensional
matrices.
( 2
min )
Transformer networks have emerged as the state-of-the-art approach for
natural language processing tasks and are gaining popularity in other domains
such as computer vision and audio processing. However, the efficient hardware
acceleration of transformer models poses new challenges due to their high
arithmetic intensities, large memory requirements, and complex dataflow
dependencies. In this work, we propose ITA, a novel accelerator architecture
for transformers and related models that targets efficient inference on
embedded systems by exploiting 8-bit quantization and an innovative softmax
implementation that operates exclusively on integer values. By computing
on-the-fly in streaming mode, our softmax implementation minimizes data
movement and energy consumption. ITA achieves competitive energy efficiency
with respect to state-of-the-art transformer accelerators with 16.9 TOPS/W,
while outperforming them in area efficiency with 5.93 TOPS/mm$^2$ in 22 nm
fully-depleted silicon-on-insulator technology at 0.8 V.
( 2
min )
Chronic stress can significantly affect physical and mental health. The
advent of wearable technology allows for the tracking of physiological signals,
potentially leading to innovative stress prediction and intervention methods.
However, challenges such as label scarcity and data heterogeneity render stress
prediction difficult in practice. To counter these issues, we have developed a
multimodal personalized stress prediction system using wearable biosignal data.
We employ self-supervised learning (SSL) to pre-train the models on each
subject's data, allowing the models to learn the baseline dynamics of the
participant's biosignals prior to fine-tuning the stress prediction task. We
test our model on the Wearable Stress and Affect Detection (WESAD) dataset,
demonstrating that our SSL models outperform non-SSL models while utilizing
less than 5% of the annotations. These results suggest that our approach can
personalize stress prediction to each user with minimal annotations. This
paradigm has the potential to enable personalized prediction of a variety of
recurring health events using complex multimodal data streams.
( 2
min )
We envision a warehouse in which dozens of mobile robots and human pickers
work together to collect and deliver items within the warehouse. The
fundamental problem we tackle, called the order-picking problem, is how these
worker agents must coordinate their movement and actions in the warehouse to
maximise performance (e.g. order throughput). Established industry methods
using heuristic approaches require large engineering efforts to optimise for
innately variable warehouse configurations. In contrast, multi-agent
reinforcement learning (MARL) can be flexibly applied to diverse warehouse
configurations (e.g. size, layout, number/types of workers, item replenishment
frequency), as the agents learn through experience how to optimally cooperate
with one another. We develop hierarchical MARL algorithms in which a manager
assigns goals to worker agents, and the policies of the manager and workers are
co-trained toward maximising a global objective (e.g. pick rate). Our
hierarchical algorithms achieve significant gains in sample efficiency and
overall pick rates over baseline MARL algorithms in diverse warehouse
configurations, and substantially outperform two established industry
heuristics for order-picking systems.
( 2
min )
Transfer learning leverages feature representations of deep neural networks
(DNNs) pretrained on source tasks with rich data to empower effective
finetuning on downstream tasks. However, the pretrained models are often
prohibitively large for delivering generalizable representations, which limits
their deployment on edge devices with constrained resources. To close this gap,
we propose a new transfer learning pipeline, which leverages our finding that
robust tickets can transfer better, i.e., subnetworks drawn with properly
induced adversarial robustness can win better transferability over vanilla
lottery ticket subnetworks. Extensive experiments and ablation studies validate
that our proposed transfer learning pipeline can achieve enhanced
accuracy-sparsity trade-offs across both diverse downstream tasks and sparsity
patterns, further enriching the lottery ticket hypothesis.
( 2
min )
Ensuring the trustworthiness and interpretability of machine learning models
is critical to their deployment in real-world applications. Feature attribution
methods have gained significant attention, which provide local explanations of
model predictions by attributing importance to individual input features. This
study examines the generalization of feature attributions across various deep
learning architectures, such as convolutional neural networks (CNNs) and vision
transformers. We aim to assess the feasibility of utilizing a feature
attribution method as a future detector and examine how these features can be
harmonized across multiple models employing distinct architectures but trained
on the same data distribution. By exploring this harmonization, we aim to
develop a more coherent and optimistic understanding of feature attributions,
enhancing the consistency of local explanations across diverse deep-learning
models. Our findings highlight the potential for harmonized feature attribution
methods to improve interpretability and foster trust in machine learning
applications, regardless of the underlying architecture.
( 2
min )
How do language models "think"? This paper formulates a probabilistic
cognitive model called the bounded pragmatic speaker, which can characterize
the operation of different variations of language models. Specifically, we
demonstrate that large language models fine-tuned with reinforcement learning
from human feedback (Ouyang et al., 2022) embody a model of thought that
conceptually resembles a fast-and-slow model (Kahneman, 2011), which
psychologists have attributed to humans. We discuss the limitations of
reinforcement learning from human feedback as a fast-and-slow model of thought
and propose avenues for expanding this framework. In essence, our research
highlights the value of adopting a cognitive probabilistic modeling approach to
gain insights into the comprehension, evaluation, and advancement of language
models.
( 2
min )
Complex diseases are caused by a multitude of factors that may differ between
patients even within the same diagnostic category. A few underlying root causes
may nevertheless initiate the development of disease within each patient. We
therefore focus on identifying patient-specific root causes of disease, which
we equate to the sample-specific predictivity of the exogenous error terms in a
structural equation model. We generalize from the linear setting to the
heteroscedastic noise model where $Y = m(X) + \varepsilon\sigma(X)$ with
non-linear functions $m(X)$ and $\sigma(X)$ representing the conditional mean
and mean absolute deviation, respectively. This model preserves identifiability
but introduces non-trivial challenges that require a customized algorithm
called Generalized Root Causal Inference (GRCI) to extract the error terms
correctly. GRCI recovers patient-specific root causes more accurately than
existing alternatives.
( 2
min )
Federated learning methods enable model training across distributed data
sources without data leaving their original locations and have gained
increasing interest in various fields. However, existing approaches are
limited, excluding many structured probabilistic models. We present a general
and elegant solution based on structured variational inference, widely used in
Bayesian machine learning, adapted for the federated setting. Additionally, we
provide a communication-efficient variant analogous to the canonical FedAvg
algorithm. The proposed algorithms' effectiveness is demonstrated, and their
performance is compared with hierarchical Bayesian neural networks and topic
models.
( 2
min )
In this paper, we investigate the training process of generative networks
that use a type of probability density distance named particle-based distance
as the objective function, e.g. MMD GAN, Cram\'er GAN, EIEG GAN. However, these
GANs often suffer from the problem of unstable training. In this paper, we
analyze the stability of the training process of these GANs from the
perspective of probability density dynamics. In our framework, we regard the
discriminator $D$ in these GANs as a feature transformation mapping that maps
high dimensional data into a feature space, while the generator $G$ maps random
variables to samples that resemble real data in terms of feature space. This
perspective enables us to perform stability analysis for the training of GANs
using the Wasserstein gradient flow of the probability density function. We
find that the training process of the discriminator is usually unstable due to
the formulation of $\min_G \max_D E(G, D)$ in GANs. To address this issue, we
add a stabilizing term in the discriminator loss function. We conduct
experiments to validate our stability analysis and stabilizing method.
( 2
min )
Despite of achieving great success in real-world applications, Deep
Reinforcement Learning (DRL) is still suffering from three critical issues,
i.e., data efficiency, lack of the interpretability and transferability. Recent
research shows that embedding symbolic knowledge into DRL is promising in
addressing those challenges. Inspired by this, we introduce a novel deep
reinforcement learning framework with symbolic options. Our framework features
a loop training procedure, which enables guiding the improvement of policy by
planning with planning models (including action models and hierarchical task
network models) and symbolic options learned from interactive trajectories
automatically. The learned symbolic options alleviate the dense requirement of
expert domain knowledge and provide inherent interpretability of policies.
Moreover, the transferability and data efficiency can be further improved by
planning with the symbolic planning models. To validate the effectiveness of
our framework, we conduct experiments on two domains, Montezuma's Revenge and
Office World, respectively. The results demonstrate the comparable performance,
improved data efficiency, interpretability and transferability.
( 2
min )
We present AIRS: Automatic Intrinsic Reward Shaping that intelligently and
adaptively provides high-quality intrinsic rewards to enhance exploration in
reinforcement learning (RL). More specifically, AIRS selects shaping function
from a predefined set based on the estimated task return in real-time,
providing reliable exploration incentives and alleviating the biased objective
problem. Moreover, we develop an intrinsic reward toolkit to provide efficient
and reliable implementations of diverse intrinsic reward approaches. We test
AIRS on various tasks of MiniGrid, Procgen, and DeepMind Control Suite.
Extensive simulation demonstrates that AIRS can outperform the benchmarking
schemes and achieve superior performance with simple architecture.
( 2
min )
In this work, we focus on the communication aspect of decentralized learning,
which involves multiple agents training a shared machine learning model using
decentralized stochastic gradient descent (D-SGD) over distributed data. In
particular, we investigate the impact of broadcast transmission and
probabilistic random access policy on the convergence performance of D-SGD,
considering the broadcast nature of wireless channels and the link dynamics in
the communication topology. Our results demonstrate that optimizing the access
probability to maximize the expected number of successful links is a highly
effective strategy for accelerating the system convergence.
( 2
min )
We introduce Breadth-First Pipeline Parallelism, a novel training schedule
which optimizes the combination of pipeline and data parallelism. Breadth-First
Pipeline Parallelism lowers training time, cost and memory usage by combining a
high GPU utilization with a small batch size per GPU, and by making use of
fully sharded data parallelism. Experimentally, we observed an increase of up
to 43% in training throughput for a 52 billion-parameter model using a small
batch size per GPU compared to Megatron-LM, which would reduce the training
time and cost by the same amount on a large GPU cluster.
( 2
min )
Animating still face images with deep generative models using a speech input
signal is an active research topic and has seen important recent progress.
However, much of the effort has been put into lip syncing and rendering quality
while the generation of natural head motion, let alone the audio-visual
correlation between head motion and speech, has often been neglected. In this
work, we propose a multi-scale audio-visual synchrony loss and a multi-scale
autoregressive GAN to better handle short and long-term correlation between
speech and the dynamics of the head and lips. In particular, we train a stack
of syncer models on multimodal input pyramids and use these models as guidance
in a multi-scale generator network to produce audio-aligned motion unfolding
over diverse time scales. Our generator operates in the facial landmark domain,
which is a standard low-dimensional head representation. The experiments show
significant improvements over the state of the art in head motion dynamics
quality and in multi-scale audio-visual synchrony both in the landmark domain
and in the image domain.
( 2
min )
Modern Deep Neural Networks (DNNs) exhibit profound efficiency and accuracy
properties. This has introduced application workloads that comprise of multiple
DNN applications, raising new challenges regarding workload distribution.
Equipped with a diverse set of accelerators, newer embedded system present
architectural heterogeneity, which current run-time controllers are unable to
fully utilize. To enable high throughput in multi-DNN workloads, such a
controller is ought to explore hundreds of thousands of possible solutions to
exploit the underlying heterogeneity. In this paper, we propose OmniBoost, a
lightweight and extensible multi-DNN manager for heterogeneous embedded
devices. We leverage stochastic space exploration and we combine it with a
highly accurate performance estimator to observe a x4.6 average throughput
boost compared to other state-of-the-art methods. The evaluation was performed
on the HiKey970 development board.
( 2
min )
The mathematical representations of data in the Spherical Harmonic (SH)
domain has recently regained increasing interest in the machine learning
community. This technical report gives an in-depth introduction to the
theoretical foundation and practical implementation of SH representations,
summarizing works on rotation invariant and equivariant features, as well as
convolutions and exact correlations of signals on spheres. In extension, these
methods are then generalized from scalar SH representations to Vectorial
Harmonics (VH), providing the same capabilities for 3d vector fields on spheres
( 2
min )
Feature-distributed data, referred to data partitioned by features and stored
across multiple computing nodes, are increasingly common in applications with a
large number of features. This paper proposes a two-stage relaxed greedy
algorithm (TSRGA) for applying multivariate linear regression to such data. The
main advantage of TSRGA is that its communication complexity does not depend on
the feature dimension, making it highly scalable to very large data sets. In
addition, for multivariate response variables, TSRGA can be used to yield
low-rank coefficient estimates. The fast convergence of TSRGA is validated by
simulation experiments. Finally, we apply the proposed TSRGA in a financial
application that leverages unstructured data from the 10-K reports,
demonstrating its usefulness in applications with many dense large-dimensional
matrices.
( 2
min )
Complex diseases are caused by a multitude of factors that may differ between
patients even within the same diagnostic category. A few underlying root causes
may nevertheless initiate the development of disease within each patient. We
therefore focus on identifying patient-specific root causes of disease, which
we equate to the sample-specific predictivity of the exogenous error terms in a
structural equation model. We generalize from the linear setting to the
heteroscedastic noise model where $Y = m(X) + \varepsilon\sigma(X)$ with
non-linear functions $m(X)$ and $\sigma(X)$ representing the conditional mean
and mean absolute deviation, respectively. This model preserves identifiability
but introduces non-trivial challenges that require a customized algorithm
called Generalized Root Causal Inference (GRCI) to extract the error terms
correctly. GRCI recovers patient-specific root causes more accurately than
existing alternatives.
( 2
min )
There is substantial empirical evidence about the success of dynamic
implementations of Hamiltonian Monte Carlo (HMC), such as the No U-Turn Sampler
(NUTS), in many challenging inference problems but theoretical results about
their behavior are scarce. The aim of this paper is to fill this gap. More
precisely, we consider a general class of MCMC algorithms we call dynamic HMC.
We show that this general framework encompasses NUTS as a particular case,
implying the invariance of the target distribution as a by-product. Second, we
establish conditions under which NUTS is irreducible and aperiodic and as a
corrolary ergodic. Under conditions similar to the ones existing for HMC, we
also show that NUTS is geometrically ergodic. Finally, we improve existing
convergence results for HMC showing that this method is ergodic without any
boundedness condition on the stepsize and the number of leapfrog steps, in the
case where the target is a perturbation of a Gaussian distribution.
( 2
min )
We investigate enhancing the sensitivity of new physics searches at the LHC
by machine learning in the case of background dominance and a high degree of
overlap between the observables for signal and background. We use two different
models, XGBoost and a deep neural network, to exploit correlations between
observables and compare this approach to the traditional cut-and-count method.
We consider different methods to analyze the models' output, finding that a
template fit generally performs better than a simple cut. By means of a Shapley
decomposition, we gain additional insight into the relationship between event
kinematics and the machine learning model output. We consider a supersymmetric
scenario with a metastable sneutrino as a concrete example, but the methodology
can be applied to a much wider class of models.
( 2
min )
We propose a novel Bayesian-Optimistic Frequentist Upper Confidence Bound
(BOF-UCB) algorithm for stochastic contextual linear bandits in non-stationary
environments. This unique combination of Bayesian and frequentist principles
enhances adaptability and performance in dynamic settings. The BOF-UCB
algorithm utilizes sequential Bayesian updates to infer the posterior
distribution of the unknown regression parameter, and subsequently employs a
frequentist approach to compute the Upper Confidence Bound (UCB) by maximizing
the expected reward over the posterior distribution. We provide theoretical
guarantees of BOF-UCB's performance and demonstrate its effectiveness in
balancing exploration and exploitation on synthetic datasets and classical
control tasks in a reinforcement learning setting. Our results show that
BOF-UCB outperforms existing methods, making it a promising solution for
sequential decision-making in non-stationary environments.
( 2
min )
ChatGPT is a large language model developed by OpenAI. Despite its impressive
performance across various tasks, no prior work has investigated its capability
in the biomedical domain yet. To this end, this paper aims to evaluate the
performance of ChatGPT on various benchmark biomedical tasks, such as relation
extraction, document classification, question answering, and summarization. To
the best of our knowledge, this is the first work that conducts an extensive
evaluation of ChatGPT in the biomedical domain. Interestingly, we find based on
our evaluation that in biomedical datasets that have smaller training sets,
zero-shot ChatGPT even outperforms the state-of-the-art fine-tuned generative
transformer models, such as BioGPT and BioBART. This suggests that ChatGPT's
pre-training on large text corpora makes it quite specialized even in the
biomedical domain. Our findings demonstrate that ChatGPT has the potential to
be a valuable tool for various tasks in the biomedical domain that lack large
annotated data.
( 2
min )
Distributional Graphormer, Microsoft’s new deep learning framework for predicting the equilibrium distribution of molecular structures, can generate realistic and diverse molecular structures with high efficiency and low cost.
The post Distributional Graphormer: Toward equilibrium distribution prediction for molecular systems appeared first on Microsoft Research.
( 14
min )
This AI system only needs a small amount of data to predict molecular properties, which could speed up drug discovery and material development.
( 9
min )
One of the essential components of deep learning is the choice of the loss
function and performance metrics used to train and evaluate models. This paper
reviews the most prevalent loss functions and performance measurements in deep
learning. We examine the benefits and limits of each technique and illustrate
their application to various deep-learning problems. Our review aims to give a
comprehensive picture of the different loss functions and performance
indicators used in the most common deep learning tasks and help practitioners
choose the best method for their specific task.
( 2
min )
Human language acquisition is an efficient, supervised, and continual
process. In this work, we took inspiration from how human babies acquire their
first language, and developed a computational process for word acquisition
through comparative learning. Motivated by cognitive findings, we generated a
small dataset that enables the computation models to compare the similarities
and differences of various attributes, learn to filter out and extract the
common information for each shared linguistic label. We frame the acquisition
of words as not only the information filtration process, but also as
representation-symbol mapping. This procedure does not involve a fixed
vocabulary size, nor a discriminative objective, and allows the models to
continually learn more concepts efficiently. Our results in controlled
experiments have shown the potential of this approach for efficient continual
learning of grounded words.
( 2
min )
This paper introduces Track Mix, a personalized playlist generation system
released in 2022 on the music streaming service Deezer. Track Mix automatically
generates "mix" playlists inspired by initial music tracks, allowing users to
discover music similar to their favorite content. To generate these mixes, we
consider a Transformer model trained on millions of track sequences from user
playlists. In light of the growing popularity of Transformers in recent years,
we analyze the advantages, drawbacks, and technical challenges of using such a
model for mix generation on the service, compared to a more traditional
collaborative filtering approach. Since its release, Track Mix has been
generating playlists for millions of users daily, enhancing their music
discovery experience on Deezer.
( 2
min )
Over the course of the past two decades, a substantial body of research has
substantiated the viability of utilising cardiac signals as a biometric
modality. This paper presents a novel approach for patient identification in
healthcare systems using electrocardiogram signals. A convolutional neural
network is used to classify users based on images extracted from ECG signals.
The proposed identification system is evaluated in multiple databases,
providing a comprehensive understanding of its potential in real-world
scenarios. The impact of Cardiovascular Diseases on generic user identification
has been largely overlooked in previous studies. The presented method takes
into account the cardiovascular condition of the patients, ensuring that the
results obtained are not biased or limited. Furthermore, the results obtained
are consistent and reliable, with lower error rates and higher accuracy
metrics, as demonstrated through extensive experimentation. All these features
make the proposed method a valuable contribution to the field of patient
identification in healthcare systems, and make it a strong contender for
practical applications.
( 2
min )
Coding problems are problems that require a solution in the form of a
computer program. Coding problems are popular among students and professionals
as it enhances their skills and career opportunities. An AI system that would
help those who practice coding problems would be highly useful and there is a
huge potential for such a system. In this work, we propose a model which uses
stacking of hyperparameter tuned boosting models to achieve impressive metric
scores of 77.8% accuracy and 0.815 PR-AUC on the dataset that was scraped from
Codeforces and Leetcode. We open source the dataset and the models developed
for this work.
( 2
min )
In this paper, we propose a novel tag-based recommender system called PLIERS,
which relies on the assumption that users are mainly interested in items and
tags with similar popularity to those they already own. PLIERS is aimed at
reaching a good tradeoff between algorithmic complexity and the level of
personalization of recommended items. To evaluate PLIERS, we performed a set of
experiments on real OSN datasets, demonstrating that it outperforms
state-of-the-art solutions in terms of personalization, relevance, and novelty
of recommendations.
( 2
min )
We propose a kernel-spectral embedding algorithm for learning low-dimensional
nonlinear structures from high-dimensional and noisy observations, where the
datasets are assumed to be sampled from an intrinsically low-dimensional
manifold and corrupted by high-dimensional noise. The algorithm employs an
adaptive bandwidth selection procedure which does not rely on prior knowledge
of the underlying manifold. The obtained low-dimensional embeddings can be
further utilized for downstream purposes such as data visualization, clustering
and prediction. Our method is theoretically justified and practically
interpretable. Specifically, we establish the convergence of the final
embeddings to their noiseless counterparts when the dimension and size of the
samples are comparably large, and characterize the effect of the
signal-to-noise ratio on the rate of convergence and phase transition. We also
prove convergence of the embeddings to the eigenfunctions of an integral
operator defined by the kernel map of some reproducing kernel Hilbert space
capturing the underlying nonlinear structures. Numerical simulations and
analysis of three real datasets show the superior empirical performance of the
proposed method, compared to many existing methods, on learning various
manifolds in diverse applications.
( 2
min )
We provide the first finite-particle convergence rate for Stein variational
gradient descent (SVGD), a popular algorithm for approximating a probability
distribution with a collection of particles. Specifically, whenever the target
distribution is sub-Gaussian with a Lipschitz score, SVGD with n particles and
an appropriate step size sequence drives the kernel Stein discrepancy to zero
at an order 1/sqrt(log log n) rate. We suspect that the dependence on n can be
improved, and we hope that our explicit, non-asymptotic proof strategy will
serve as a template for future refinements.
( 2
min )
In this paper, we consider a general observation model for restless
multi-armed bandit problems. The operation of the player needs to be based on
certain feedback mechanism that is error-prone due to resource constraints or
environmental or intrinsic noises. By establishing a general probabilistic
model for dynamics of feedback/observation, we formulate the problem as a
restless bandit with a countable belief state space starting from an arbitrary
initial belief (a priori information). We apply the achievable region method
with partial conservation law (PCL) to the infinite-state problem and analyze
its indexability and priority index (Whittle index). Finally, we propose an
approximation process to transform the problem into which the AG algorithm of
Ni\~no-Mora and Bertsimas for finite-state problems can be applied to.
Numerical experiments show that our algorithm has an excellent performance.
( 2
min )
The paper uses structured machine learning regressions for nowcasting with
panel data consisting of series sampled at different frequencies. Motivated by
the problem of predicting corporate earnings for a large cross-section of firms
with macroeconomic, financial, and news time series sampled at different
frequencies, we focus on the sparse-group LASSO regularization which can take
advantage of the mixed frequency time series panel data structures. Our
empirical results show the superior performance of our machine learning panel
data regression models over analysts' predictions, forecast combinations,
firm-specific time series regression models, and standard machine learning
methods.
( 2
min )
Gaussian Processes (GPs) offer an attractive method for regression over
small, structured and correlated datasets. However, their deployment is
hindered by computational costs and limited guidelines on how to apply GPs
beyond simple low-dimensional datasets. We propose a framework to identify the
suitability of GPs to a given problem and how to set up a robust and
well-specified GP model. The guidelines formalise the decisions of experienced
GP practitioners, with an emphasis on kernel design and options for
computational scalability. The framework is then applied to a case study of
glacier elevation change yielding more accurate results at test time.
( 2
min )
BioAutoMATED, an open-source, automated machine-learning platform, aims to help democratize artificial intelligence for research labs.
( 8
min )
Amazon SageMaker is an end-to-end machine learning (ML) platform with wide-ranging features to ingest, transform, and measure bias in data, and train, deploy, and manage models in production with best-in-class compute and services such as Amazon SageMaker Data Wrangler, Amazon SageMaker Studio, Amazon SageMaker Canvas, Amazon SageMaker Model Registry, Amazon SageMaker Feature Store, Amazon SageMaker […]
( 8
min )
A diverse range of artists, fashionistas, musicians and the cinematic arts inspired the creative journey of Pedro Soares, aka Blendeered, and helped him fall in love with using 3D to create art.
( 7
min )
GFN Thursday arrives alongside the sweet Steam Summer Sale — with hundreds of PC games playable on GeForce NOW available during Valve’s special event for PC gamers. Also on sale, OCTOPATH TRAVELER and OCTOPATH TRAVELER II join the GeForce NOW library as a part of five new games coming to the service this week. Saved Read article >
( 6
min )
A global team of medical providers is leveraging Holoportation, a Microsoft 3D capture and communication technology, to widen access to specialized care. Computer engineer Spencer Fowers and plastic surgeon Kwame Darko discuss the collaboration.
The post Collaborators: Holoportation™ communication technology with Spencer Fowers and Kwame Darko appeared first on Microsoft Research.
( 32
min )
No content preview
( 2
min )
GPT-3.5 Turbo, DALL·E and Whisper APIs are also generally available, and we are releasing a deprecation plan for older models of the Completions API, which will retire at the beginning of 2024.
( 5
min )
Amazon Polly is a service that turns text into lifelike speech. It enables the development of a whole class of applications that can convert text into speech in multiple languages. This service can be used by chatbots, audio books, and other text-to-speech applications in conjunction with other AWS AI or machine learning (ML) services. For […]
( 9
min )
Predictive maintenance is critical in automotive industries because it can avoid out-of-the-blue mechanical failures and reactive maintenance activities that disrupt operations. By predicting vehicle failures and scheduling maintenance and repairs, you’ll reduce downtime, improve safety, and boost productivity levels. What if we could apply deep learning techniques to common areas that drive vehicle failures, unplanned […]
( 11
min )
In the dynamic world of the Internet of Things (IoT), data integration plays a crucial role in harnessing the full potential of connected devices. By seamlessly combining data from diverse sources, data integration enables organizations to unlock valuable insights, optimize operations, and make informed decisions. This blog will explore the significance of data integration in… Read More »Data integration in IoT environments: Enhancing connectivity and insights
The post Data integration in IoT environments: Enhancing connectivity and insights appeared first on Data Science Central.
( 21
min )
This is the first in a series of articles based on interviews with Intel technology leaders about AI/HPC acceleration.
The post Ushering in the 5th epoch of distributed computing with accelerated AI technologies appeared first on Data Science Central.
( 34
min )
In the movie the Lord of the Rings – the wizard Sauron says that “The hour is later than you think” I was reminded of this phrase when I read a report from McKinsey The economic potential of generative A There are some key findings on the future of AI which shows you how fast… Read More »The hour is later than you think for AI impacting our jobs
The post The hour is later than you think for AI impacting our jobs appeared first on Data Science Central.
( 19
min )
As Web3 evolves and transforms into a more decentralized and user-centric ecosystem, the role of artificial intelligence or AI cannot be understated. By leveraging its capabilities, AI is contributing to various aspects of the Web3 landscape, such as managing data, executing contracts, generating insights, securing identities, curating content, governing organizations, and enhancing user experiences. An… Read More »Role of AI in Web3: Ensuring seamless content moderation for dating websites
The post Role of AI in Web3: Ensuring seamless content moderation for dating websites appeared first on Data Science Central.
( 23
min )
Three leading European generative AI startups joined NVIDIA founder and CEO Jensen Huang this week to talk about the new era of computing. More than 500 developers, researchers, entrepreneurs and executives from across Europe and further afield packed into the Spindler and Klatt, a sleek, riverside gathering spot in Berlin. Huang started the reception by Read article >
( 6
min )
China electric vehicle maker XPENG Motors has announced its new G6 coupe SUV — featuring an NVIDIA-powered intelligent advanced driver assistance system — is now available to the China market. The G6 is XPENG’s first model featuring the company’s proprietary Smart Electric Platform Architecture (SEPA) 2.0, which aims to reduce development and manufacturing costs and Read article >
( 5
min )
The increased frequency and severity of extreme weather and climate events could take a million lives and cost $1.7 trillion annually by 2050, according to the Munich Reinsurance Company. This underscores a critical need for accurate weather forecasting, especially with the rise in severe weather occurrences such as blizzards, hurricanes and heatwaves. AI and accelerated Read article >
( 6
min )
AI and accelerated computing will help climate researchers achieve the miracles they need to achieve breakthroughs in climate research, NVIDIA founder and CEO Jensen Huang said during a keynote Monday at the Berlin Summit for the Earth Virtualization Engines initiative. “Richard Feynman once said that ‘what I can’t create, I don’t understand’ and that’s the Read article >
( 6
min )
Companies across various industries create, scan, and store large volumes of PDF documents. In many cases, the content is text-heavy and often written in a different language and requires translation. To address this, you need an automated solution to extract the contents within these PDFs and translate them quickly and cost-efficiently. Many businesses have diverse […]
( 8
min )
In computer vision (CV), adding tags to identify objects of interest or bounding boxes to locate the objects is called labeling. It’s one of the prerequisite tasks to prepare training data to train a deep learning model. Hundreds of thousands of work hours are spent generating high-quality labels from images and videos for various CV […]
( 9
min )
We focus on the problem of market making in high-frequency trading. Market
making is a critical function in financial markets that involves providing
liquidity by buying and selling assets. However, the increasing complexity of
financial markets and the high volume of data generated by tick-level trading
makes it challenging to develop effective market making strategies. To address
this challenge, we propose a deep reinforcement learning approach that fuses
tick-level data with periodic prediction signals to develop a more accurate and
robust market making strategy. Our results of market making strategies based on
different deep reinforcement learning algorithms under the simulation scenarios
and real data experiments in the cryptocurrency markets show that the proposed
framework outperforms existing methods in terms of profitability and risk
management.
( 2
min )
Biological data may be separated into primary data, such as gene expression,
and secondary data, such as pathways and protein-protein interactions. Methods
using secondary data to enhance the analysis of primary data are promising,
because secondary data have background information that is not included in
primary data. In this study, we proposed an end-to-end framework to integrally
handle secondary data to construct a classification model for primary data. We
applied this framework to cancer prognosis prediction using gene expression
data and a biological network. Cross-validation results indicated that our
model achieved higher accuracy compared with a deep neural network model
without background biological network information. Experiments conducted in
patient groups by cancer type showed improvement in ROC-area under the curve
for many groups. Visualizations of high accuracy cancer types identified
contributing genes and pathways by enrichment analysis. Known biomarkers and
novel biomarker candidates were identified through these experiments.
( 2
min )
Recently, deep learning has revolutionized the field of natural language
processing, with neural language models proving to be very effective for
next-word prediction. However, a rigorous theoretical explanation for their
success in the context of formal language theory has not yet been developed, as
it is unclear why neural language models can learn the combinatorial rules that
govern the next-word prediction task. In this paper, we study a class of formal
languages that can be used to model real-world examples of English sentences.
We construct neural language models can solve the next-word prediction task in
this context with zero error. Our proof highlights the different roles of the
embedding layer and the fully connected component within the neural language
model.
( 2
min )
We examine the characteristic activation values of individual ReLU units in
neural networks. We refer to the corresponding set for such characteristic
activation values in the input space as the characteristic activation set of a
ReLU unit. We draw an explicit connection between the characteristic activation
set and learned features in ReLU networks. This connection leads to new
insights into why various neural network normalization techniques used in
modern deep learning architectures regularize and stabilize SGD optimization.
Utilizing these insights, we propose a geometric approach to parameterize ReLU
networks for improved feature learning. We empirically verify its usefulness
with less carefully chosen initialization schemes and larger learning rates. We
report improved optimization stability, faster convergence speed, and better
generalization performance.
( 2
min )
Practical density functional theory (DFT) owes its success to the
groundbreaking work of Kohn and Sham that introduced the exact calculation of
the non-interacting kinetic energy of the electrons using an auxiliary
mean-field system. However, the full power of DFT will not be unleashed until
the exact relationship between the electron density and the non-interacting
kinetic energy is found. Various attempts have been made to approximate this
functional, similar to the exchange--correlation functional, with much less
success due to the larger contribution of kinetic energy and its more non-local
nature. In this work we propose a new and efficient regularization method to
train density functionals based on deep neural networks, with particular
interest in the kinetic-energy functional. The method is tested on
(effectively) one-dimensional systems, including the hydrogen chain,
non-interacting electrons, and atoms of the first two periods, with excellent
results. For the atomic systems, the generalizability of the regularization
method is demonstrated by training also an exchange--correlation functional,
and the contrasting nature of the two functionals is discussed from a
machine-learning perspective.
( 2
min )
A Bayesian filtering algorithm is developed for a class of state-space
systems that can be modelled via Gaussian mixtures. In general, the exact
solution to this filtering problem involves an exponential growth in the number
of mixture terms and this is handled here by utilising a Gaussian mixture
reduction step after both the time and measurement updates. In addition, a
square-root implementation of the unified algorithm is presented and this
algorithm is profiled on several simulated systems. This includes the state
estimation for two non-linear systems that are strictly outside the class
considered in this paper.
( 2
min )
Recently, a new class of non-convex optimization problems motivated by the
statistical problem of learning an acyclic directed graphical model from data
has attracted significant interest. While existing work uses standard
first-order optimization schemes to solve this problem, proving the global
optimality of such approaches has proven elusive. The difficulty lies in the
fact that unlike other non-convex problems in the literature, this problem is
not "benign", and possesses multiple spurious solutions that standard
approaches can easily get trapped in. In this paper, we prove that a simple
path-following optimization scheme globally converges to the global minimum of
the population loss in the bivariate setting.
( 2
min )
In this work, a comprehensive numerical study involving analysis and
experiments shows why a two-layer neural network has difficulties handling high
frequencies in approximation and learning when machine precision and
computation cost are important factors in real practice. In particular, the
following fundamental computational issues are investigated: (1) the best
accuracy one can achieve given a finite machine precision, (2) the computation
cost to achieve a given accuracy, and (3) stability with respect to
perturbations. The key to the study is the spectral analysis of the
corresponding Gram matrix of the activation functions which also shows how the
properties of the activation function play a role in the picture.
( 2
min )
Sponsored Post Attend the Data Science Symposium 2022 on November 8 The Center for Business Analytics at the University of Cincinnati will present its annual Data Science Symposium 2022 on November 8. This all day in-person event will have three featured speakers and two tech talk tracks with four concurrent presentations in each track. The […]
The post Attend the Data Science Symposium 2022, November 8 in Cincinnati appeared first on Machine Learning Mastery.
( 10
min )
.jpg)
